So this is more than likely a normal thing that happens. My heat shield that goes over the manifold rotted away right where the manifold joins the down pipe. So with it only 2 bolts attaching it to the block, it rattles at times.
I've tried 2 different way to stop the rattle with no luck. Removing it to bend it to press up against the manifold and the other way, bending it up away enough to have clearance but still help to cover / block the manifold.
I guess the real question now is, since the manifold is in the back on the front, would it be safe to run the car without it?
Here it is now with it bent up;