Sounds like your feeder pens are not doing their job if the pigs are getting in. If you goal is deer I would suggest you don't let the pigs into the feeder pen.
Like others said, many are having hard times baiting deer in because of all the natural food available, and that is really state wide right now.