It turns out that a hot solution of alum (ammonium aluminum sulfate), which is available in the spice rack of your local supermarket, will dissolve the steel drill bit without reacting with the brass.

There are a number of different types of drill bits. Steel Tooth Rotary Bits are the most common types of drill bits, while Insert Bits are steel tooth bit with tungsten carbide inserts.