The Turn of the Screw is a horror story that works within the Gothic tradition. Published in 1898, it is a bit too late to fit perfectly neatly the Gothic era (which was arguably at its peak from the late-18th century to the mid-19th century), but it certainly operates within that tradition—the governess even alludes to Jane Eyre and The Mysteries of Udolpho in her narrative, thus aligning The Turn of the Screw with two of the most famous Gothic novels.
On another note, The Turn of the Screw is also a novella, but it's surprisingly slow-paced for such a condensed form. Most novellas work their way rather quickly to the narrative's inciting incident and even its climax, but The Turn of the Screw uses suspense and foreshadowing to dwell in ambiguity for as long as possible. This sense of prolonging and delay, in fact, is what creates most of the horror in the novella. For that matter, the horror elements don't solely come from the material itself but rather from the uncertainty and ambiguity surrounding what actually ends up happening, as readers are forced—by the end—to second-guess their interpretations of what happened earlier in the narrative. In this regard, Henry James uses ambiguity as a form of psychological horror, knowing that what unsettles readers most is an inability to fully understand such a disturbing story, thus forcing them to use their imaginations to fill in the gaps—a disquieting mental process that can extend far beyond the pages of the book itself.