On the most rudimentary level, the setting of the story is the Giant's property and its surrounding neighborhood. Wilde does not make it entirely clear, however, what world the Giant's garden belongs to. While it is certainly imaginary (one of the main characters is a Giant, the children do not seem to age, and the plants, seasons, and weather are anthropomorphized), this world mirrors the real world to some extent. For example, the children all go to school—an unexpected routine for a fairytale set in a magical world. In addition, the Giant writes a sign saying that "TRESPASSERS WILL BE PROSECUTED," implying—with the word "prosecuted"—that the legal conventions and formulations of the real world are to some degree present in the Giant's world, too.
Usually, the weather and seasons would figure as part of a story's setting. In "The Selfish Giant," however, they behave more like characters than passive elements of the story's environment.
But the Spring never came, nor the Summer. The Autumn gave golden fruit to every garden, but to the Giant's garden she gave none. "He is too selfish," she said. So it was always Winter there, and the North Wind, and the Hail, and the Frost, and the Snow danced about through the trees.
Wilde's anthropomorphism of the weather and seasons ensures that these environmental factors both serve as characters and as a part of the story's setting. It is the brutality of the forces of winter and the boycott of Spring, Summer, and Autumn that finally push the Giant to rethink how he engages with the world. They affect him because they are a part of the setting, but they're also more active than the average setting-related detail because of the story's use of anthropomorphism.