Rafi was raised in a Jewish household in North London, the son of English-born publisher Simon Gavron and US-born writer Martha Pichey. His paternal grandfather, Robert Gavron, Baron Gavron, a millionaire, philanthropist, and Labour Life Peer, was born in London, to an Eastern European Jewish family. Rafi's paternal grandmother, Hannah Fyvel, who was a sociologist, was born in Tel Aviv, Israel, to a Jewish family that had lived in Germany, Moravia, Belarus, and Lithuania. One of Rafi's paternal great-grandfathers, Tosco R. Fyvel (born Raphael Joseph Feiwel), was a prominent author, journalist, and literary editor, who had worked with
Golda Meir and was an acquaintance of
George Orwell. Rafi's paternal grandfather was later married to
Nicky Gavron, a politician who served as Lord Mayor of London (their daughter, Rafi's half-aunt, is director
Sarah Gavron). Rafi's maternal grandfather was from a French-Canadian family.
