I suspect that Jo Rowling probably imagined James and Harry as white too, i don’t mind that, that’s her business.
Personally I’m mentally acting against the white-as-default-unless-otherwise-specified that’s pervasive in imagining media especially in predominately white fandom culture.
I’ve generally operated on a POC-unless-otherwise-noted basis with how I imagine characters. My HP headcanons aren’t What If Everybody In Harry Potter Was Black Instead Of White. They only read that way if you’re still stuck in the white-as-default zone.
James’ skin color, hair texture, etc to my knowledge has never been specified and that means to me that even a reader who strictly contains themselves within the bounds of canon is free to imagine him as any number of ethnicities.
In absence of a specified race I chose to imagine the one that makes the story most compelling to me.