Doyle sets The Sign of Four in London and, through flashback, India. The novel unfolds around the time of Doyle's writing, in the late Victorian period (around 1890). London, full of fog, rain, and pollution, is the perfect backdrop for a mystery:
It was a September evening and not yet seven o'clock, but the day had been a dreary one, and a dense drizzly fog lay low upon the great city. Mud-coloured clouds drooped sadly over the muddy streets. Down the Strand the lamps were but misty splotches of diffused light which threw a feeble circular glimmer upon the slimy pavement. The yellow glare from the shop-windows streamed out into the steamy, vaporous air and threw a murky, shifting radiance across the crowded thoroughfare.
While the city, in such a description, appears as a sinister, insular environment that keeps no shortage of secrets, neither London nor the events that unfold there are immune to the larger political forces of Britain at the time. The British Empire's expansion—and, in particular, the brutal regime of the British Raj, or the British government in colonial India—form a vital historical context to the novel. For one, the Agra treasure at the center of the novel's mystery is a symbol of the British extraction of natural wealth from India. Additionally, the racist attitude that Doyle expresses—in the body of the narrative and in the sentiments of his characters—toward Black people is emblematic of the racialized power structures that the British Empire depended on in order to justify its dominion.