A painting rendered in real time.
The starting point is Ivan Shishkin's Rain in an Oak Forest (1891): soft rain, mist and muted earth tones, with a mood open enough that viewers bring their own feeling to it. Landscape painting often uses weather this way, for example rain for grief or storm for rage. Ruskin called this the pathetic fallacy and considered it a fault.
In this real-time version, the viewer changes the weather themselves.
The canvas inside the scene
A canvas stands in the landscape, showing a real-time rendered artwork blended with the oil painting. As the weather changes, the material reads the current state and blends between the sunny painting, the cloudy painting and the live scene with a gradual dissolve.
The technical and visual choices were made together: ray-marched cloud, surfaces that get wet and dry again, and foliage that shows the wind.
Weather as a narrative device.
The project combines ideas from classical painting with real-time graphics, with a focus on weather and emotion.
Generated images and the original painting
The project also asks how an image generated by a program relates to a painting, and how a re-creation relates to the work it comes from. The landscape is computed in real time from shaders and simulated light, yet it is composed and lit after classical landscape painting. The canvas in the scene holds Shishkin's painting and the live render together and dissolves from one to the other as the weather changes, so the original and its re-creation can be compared in a single frame. Because the viewer sets the weather, each viewing produces a different version of the painting.
Theory
- Multimodal analysis (Burn, 2016). Weather works as a symbolic narrative tool, turning light, cloud density and colour shift into emotional metaphor.
- Reader-response theory (Ryan, 2003). Media is interpreted through personal experience, so the project focuses on how individual players read a dynamic weather system, with no single intended meaning.
Case studies
Rain in an Oak Forest (Shishkin, 1891)
Soft rain, misty forest, muted earthy tones: calmness, solitude, introspection. Open-ended enough that viewers project their own feeling onto it.
Red Dead Redemption 2 (Rockstar, 2018)
Rain-soaked ground, mist on distant mountains, ambient rain. Mechanics like slowed movement during storms tie the mood to how the player moves.
Technical research
- Ray marching for volumetric cloud, fog and heavy rain.
- Procedural shaders for surface wetness, puddles and water refraction that respond to conditions.
- Niagara for rain, lightning and weather transitions, synchronised with events in the scene.
- Adaptive sound keyed to weather intensity and player location.
Design and rendering.
Artistic vision
The design draws from classical landscape painting. Weather transitions are rendered with shifting tone, light intensity and volumetric depth, and each weather pattern carries a narrative role: melancholy in rain, tension in storm, tranquility in clear sky.
Choosing the weather
Rain, fog and storm were selected for their symbolic connection to sadness, uncertainty and tension, and for the visual contrast they offer. Realism is exaggerated where it helps the mood, and transitions are blended so nothing snaps.
Environmental composition
- Modular landscapes: trees, rocks and water features assembled so the world can shift with the emotional arc.
- Day and night cycle: an evolving time-of-day system moves ambient lighting and skybox together, so light and shadow guide the viewer through the change.
- Open exploration: the viewer chooses when the weather changes.
Technical art as the artwork
Shader programming and volumetric lighting carry the visual narrative directly. Real-time pipelines, LOD management and optimised Niagara effects keep it running at frame rate.
Technical appendix: systems, shaders and pipeline
System overview.
Weather control
- Input: two sliders: weather (sunny / cloudy / rainy) and time (day / night).
- Clouds: colour, density and movement adjusted through the cloud manager.
- Terrain: textures, puddles and foliage react to the current state.
- Rain and thunder: handled by Niagara.
- Lighting: environment and landscape lighting blended so one condition slides into the next.
Volumetric cloud
Ray marching calculates the path of light through volumetric data, allowing dynamic density, lighting and movement. Skies vary, and light passing through cloud produces effects such as Tyndall beams.
Dynamic terrain
- Puddles: the most tangible sign of rain, sized dynamically so that raining, just-rained and sunny all read differently on the ground.
- Foliage: wind cannot be seen directly, so foliage blended through a vertex shader makes it visible.
- Slope-aware texturing: terrain vectors decide whether an area is flat or sloped and select texture accordingly, with lerped transitions at the intersection.
Rain and lightning
CPU-based rain handles the foreground so collision detection works on interactive elements; GPU-based rain covers distance and background to keep the cost down. Lightning went through several iterations. The first pass was only a bolt, so a screen-space point light was added to flash the whole scene, and thunder plays one to three seconds after the strike.
Real-time render as generated art
A canvas in the scene displays a real-time rendered artwork blended with fine art. The blended material reads the current weather value from the UI and lerps between the sunny painting, the cloudy painting and the live scene, so the painting changes gradually with the sky.
Testing and iteration
Changes and next steps.
- Colour correction: presets adjusted against real-life weather reference in video and photography.
- Screen raindrop layer: a glass-rain effect so the viewer sees drops running down the lens, changing as the rain gets heavier.
- Light and environment mixer: further tuning of settings to make the ambience more vivid.
Further development
The current art integration works by algorithmic overlay and blending. A stronger solution would analyse the work of individual painters to build per-painter material functions and style presets, then generate new work from those.
Further work could add sessions where participants use the simulator while their emotional responses are recorded through participant observation, along with more weather variations and a way for the player to paint.
Stills.
