Part of my server decomission process is to defect that server from its MSX server. Either that step got skipped or I didn’t put it in my instructions. Either is entirely possible. By the time I noticed, the server had already been shut down and the VM dropped. And I had an alert firing that was telling me that this server wasn’t polling. And this makes perfect sense.
Let me say that I hate hacking at system tables. It’s verboten with the master database in my environment unless it goes through a review process. And today, I had a alert firing off with immediate escalation because a server was down.
We learned two lessons here. First, make sure you fully defect your target from its MSX server before shutting it down and dropping the VM. Second, we now know how to do this manually in case it happens again. Target servers live in msdb.dbo.systargetservers. And a simple delete statement made it go away forever.
delete from msdb.dbo.systargetservers where server_id = 22
I fear there will probably be some remnants of that server in downstream tables because there were no constraints nor foreign keys on that table. I’ll have to clean those up next.