So the general consensus on the forums is that the VP system is currently broken!!
To get bigger and better barter rewards (silver for your camp) you should barter with higher level camps.
Now for the broken bit: to get the higher VP possible, you should action against lower level camps!
In addition, to gain the max bonus % for getting spoils, you should always place your action against the camp that you have highest. For example, if your 'White Knife' camp has the highest upgrade level, always action against opponents 'White Knife' camps for your best chance of spoils.