I've been mulling over this topic since Rachel posted on using other cultures in fantasies.
I'm not going to limit myself to writing about middle aged white women in the American Midwest, especially since I write science fiction and fantasy. The whole point of SF is to escape limits. So I have to write about other people and cultures.
I also think of myself as an affirmative action author. I make a point of including all kinds of people in my stories and of drawing on many human cultures.
My story "Knapsack Poems" is based directly on Basho's Journey to the North, though I added a couple of incidents from the Egils saga Skallagrimssonar.
The hwarhath in Ring of Swords are a mixture of viking and Confucian traits, and much of their culture is based on the culture on Shogunate Japan: all those lumpy pots and gorgeous brocade gowns and the plays, which are based on Noh. I read every translation of Noh plays I could find, when I was writing Ring of Swords.
However, once you write directly about other people and cultures, not disguising them as aliens, you lay yourself open to charges of ignorance and prejudice.
I have tended to hedge my bets. Most of my characters are mixed culture, so it's hard to argue that I have gotten their culture wrong. There are a few exceptions: Eddie Whirlwind in A Woman of the Iron People was my attempt to describe a pissed off Native American academic. One of the reviews said no one could hold a grudge as long as the characters in my novel. I think the reviewer was talking about Eddie.
I figure, you does your research,and you takes your chances. Anything is better than a white bread future or fantasy.
I will point out that the writers of generic fantasies do not know the European cultures they supposedly draw on. They are imitating Tolkien or imitators of Tolkien, rather than going back to the original sources. At this point, generic fantasy has its own culture, which part Tolkien, part fairy tale, part D&D and part Hollywood versions of the middle ages.
There are two issues here. One is setting fantasies in non-European cultures, always worth doing. But the other is creating fantasy worlds that are not based on a specific historical culture, but draw on something besides the generic fantasy world. There is no reason why the people in a fantasy need to be white. They aren't in the Earthsea Trilogy. There is no reason why the world of a fantasy needs to be Hollywood Euro-medieval.