I just read through the latter half of the journal of a boiler engineer on the USS Nicholas (Fletcher-class DDE) and given how many times "Alongside [fleet oiler] to fuel the boat" appeared, I don't think you need to worry much about her being pulled from the front.
But, then again, our subs were kinda tough on the Japanese merchant marine...

There's an IJN replenish ship named Kitakami Maru.

There were lots of Japanese replenishment ships, as far as I know. What I don't know is whether they sailed with them in the midst of their task forces like we did. The Japanese seemed more inclined to run small skirmishing fleets when compared to our massive fleet movements. But, it's not something I've studied.

Oh, I wasn't actually talking about IJN replenishment ships, I'm just telling that there was a replenishment ship that is also named Kitakami since they are talking about recharging (replenishing).

Anyway, support ships like fleet tankers and supply ships really did sailed together with their main force during several engagements, like during the Attack on Pearl Harbor, 8 tankers were present to support the Kido Butai while escorted by destroyers. Though I never heard any support ships from both sides that were sent to the front-line of an actual combat (though there are support ships that were sank during engagements) if the USN did do such a thing, probably the support ships were escorted by their DDs and DDEs that are properly armed with anti-submarine and anti-aircraft weapons.

If you're interested, here's a list of Japanese oil tankers and the records of their movements during the war. I also remembered that you're the one who give me that site in the first place.

OOZ662 said:

There were lots of Japanese replenishment ships, as far as I know. What I don't know is whether they sailed with them in the midst of their task forces like we did. The Japanese seemed more inclined to run small skirmishing fleets when compared to our massive fleet movements. But, it's not something I've studied.

On raids and large operations, yes. They rendezvous with their oilers on certain points, usually they did that after an operation and not before it, I think.