In the short haul intermodal business, and anything East of the Misississippi River moving East-West is short haul, you need a relative balance of loads in both direction to make service viable. Now, on moving containers to the Far East, the steamship lines owning the containers would gladly pay the railroad or a trucker to move empties from Kearny to Delmar rather than Kearny to Long Beach, CA, if a load were available on the Eastern Shore.
Obviously, the dollars don't add up though, or someone would already be doing it in reality. Or maybe they're just trucking to the nearest ramp (Baltimore?).