I have a similar setup, with one difference.
I drill a thru hole for the actuator (about the inside diameter of the brass tube).
Then enlarge the hole for the brass tube only through the foam. That way the tube
will not fall thru and just a touch of glue will hold it in place.
It's been working well me.