The core issue, as it always is in such matters, is that the WM, although built with more modern technology and engineering (i.e. low grades, broad clearances, tunnels and bridges) it was more expensive to maintain (i.e. tunnels and bridges) The B&O was there first, so it hugged the river's edge longer and had to climb more steeply to cross the ridges. The WM came some decades later, and opted for the higher ground. This reduced curvature and grade, making it cheaper to operate, but it required said bridges and tunnels, so it was more expensive to take care of. Since the WM used the route primarily for profitable thru per diem traffic and low cost bulk coal and grain moving to port, it worked for them. Very little expensive switching and local work along the way.
The bean counters in the Terminal Tower recognized that more trains could be run over the B&O's double tracked route, and thus could more readily absorb the WM's traffic, which was mostly thru movements with very little on-line business. To absorb the B&O's traffic over the WM, massive investment would have been needed to double track it, upgrade the terminals, and somehow deal with B&Os local on-line service on the route. So keeping it in-house wasn't really an option.
It would have been nice if the ICC forced Chessie to divest itself of the route, making it available for a competitor like NW or PLE to access the Port of Baltimore, but they were busy sifting through the rubble of the Penn Central at the time, and couldn't be bothered with such trivia.
As it is, I believe the WM's right of way is the obvious choice to create a high speed rail corridor between Washington and Chicago, at least from Cherry Run to the old PLE yard at Rook. There are a couple of notable chunks of right of way that have been interrupted, but the infrastructure through the Potomac, Casselman and Youghiogheny valleys is already set up to far exceed 100 mph, The bridges and tunnels need to be rehabbed, but probably not entirely replaced.
The biggest obstacle would be the bicyclists... the same lot that fought to preserve the ROW as a rail trail, "Just in case" it needed to be put back into use as a railroad. The trails have become so popular, that they will likely force high speed rail to carve an entirely new ROW out of private property, pristine environments and sensitive habitats. I'm pretty sure I won't see any of that in my lifetime.
That's my story and I'm sticking to it.
Lee