It occurred to me that, perhaps, Raspbian headless might be as minimal as one can get without giving up some crucial functionalities unique to Raspbian/Raspberry Pi
This depends what you mean by "minimal". If you mean, system load at boot, which is essentially about RAM and CPU usage when idle, then it is pretty close to a minimum. Getting nominal CPU usage (<1%) with an idle system is I think standard on any general purpose computer running a sane OS sans GUI, so the most significant issue is then just RAM.
I run Pi's headless with Raspbian or Fedora; with Raspbian I start from jessie-lite and just disable a few of the init services that I don't care about (I do pretty much the same with Fedora, although of course it requires some other work for use on a Pi).
Generally my RAM usage on a base system like this is 20-25 MiB. I've never tried to get it down less than that because it seems slender enough considering this is still really a general purpose computer running a normative, contemporary general purpose OS (in fact, I boot to console on my big linux machines and they can start at about the same point).
My general point is there's not much you need to do with jessie-lite if that's what you want.
However, if by "minimal" you mean the install size, I am sure there are a few things you could get rid of. Maybe you could even cut that 2 GB in half -- but this will not impact or improve the system performance. It will just save you some space on the card...until you start installing stuff, at which point I think you will find many of the "optional" things you got rid of sneak back in as dependencies.