Depends on what your riding plans are: is your primary concern how the bike rides fully loaded, or are you more concerned on how the bike performs once you have unloaded it?
If the former, then set sag and damping rates for a fully loaded bike and expect that the suspension will feel harsh when bike is unloaded.
If the latter, then keep sag and damping where they are now and expect the bike to feel soft when fully loaded.
Or, if both riding conditions are important, you can strike a happy medium between the two by setting sag and damping to the average of the above two scenarios.