He would shoot in for a takedown, but I would sprawl or otherwise (immediately) reverse it so that I ended up on top. I was not attempting takedowns, only defending.
The second guy, I did go for a few takedowns after he initiated. I have to admit that defending takedowns takes almost as much energy as free-rolling! I was tired at the end there!