Well I'm no expert, but 4mm sounds like it should be do-able for drill and tap? Sweating is commonly used so not sure why it should wreck the barrel (other than the finish). Failing all that, have a look at Loctite Black Maxx adhesive. I've never used it myself, but have read several times of it being used for attaching sights with good results.
Bookmarks