Emotional Weather

An integrated real-time rendered weather simulator based on unreal engine and Media Research

My Role

Researcher
Media Artist
Programmer

Art Project
CG Programming
Interactive Media Design

Associated with UCL

Project Overview

Sunny- Good Mood

Moon Night- Thoughtful Emotion

God Light- Appreciation

Project Development Process

01 Theory
Case Study
Tech Research

02 Artistic Vision
Env. Composition
Tech Art Integration

03 System
Day-Night Shift
Volumetric Cloud
Dynamic Terrain
VFX/Data Assets/UI
Art Generation

04 Further Development

Research

The Emotional Weather project explores how real-time weather systems in video games influence player emotions by merging classical art concepts with advanced computer graphics technology. This research-driven approach blends artistic inspiration with cutting-edge technical solutions, creating immersive and emotionally engaging experiences.

Theoretical Framework

01

The theoretical foundation of Emotional Weather combines media theory with user-centered interpretation, offering insights into how weather can act as both a narrative device and an emotional catalyst.

1

Why Media Theory and Semiotics Matter

Multimodal analysis (Burn, 2016) highlights how weather can function as a symbolic narrative tool, transforming visual elements like light, cloud density, and color shifts into emotional metaphors. These design choices create layered emotional landscapes that resonate with users on a sensory and symbolic level.

How Does Reader-Response Theory Fit In?

Reader-response theory (Ryan, 2003) suggests that media is interpreted through the lens of personal experience. This approach influenced the project’s focus on how individual players emotionally interact with and interpret dynamic weather systems in games.

Case Studies

02

Art and interactive media offer powerful platforms for exploring the emotional resonance of weather. Through classical painting and modern game design, this project examines how visual and auditory elements, combined with interactivity, can evoke profound emotional responses.

Painting: Rain in an Oak Forest (Ivan Shishkin, 1891)

Visual Elements: Soft rain, misty forests, and muted earthy tones evoke calmness, solitude, and introspection.
Emotional Impact: Open-ended interpretation invites viewers to project personal emotions, demonstrating how static art can trigger diverse emotional responses.

Game: Red Dead Redemption 2 (Rockstar Games, 2019)

Visual & Auditory Design: Rain-soaked environments, distant mountains shrouded in mist, and ambient rain sounds create a melancholic, immersive atmosphere.
Interactive Effects: Mechanics such as slowed movement during storms reinforce emotional responses, highlighting how gameplay systems shape user experiences.

Technical Research

03

Shader Programming and Rendering Techniques

Ray Marching: Used to create volumetric clouds, simulating realistic weather patterns such as fog and heavy rain.
Procedural Shaders: Custom shaders simulate surface wetness, puddles, and water refraction, dynamically responding to changing weather conditions.

Real-Time Weather Simulation

Niagara Particle System: Powers dynamic rain, lightning, and weather transitions. Particle effects synchronize with game events for seamless environmental changes.
Environmental Sound Design: Adaptive sound effects, including rain, wind, and thunder, adjust based on weather intensity and player location.

Game Engine Integration

Unreal Engine 5: Utilized for real-time rendering, lighting adjustments, and environmental simulation, ensuring both technical precision and artistic expression.

Design

The Emotional Weather project transforms theoretical research into a compelling interactive experience through thoughtful media art design. This section outlines the artistic vision, interaction mechanics, and environmental design that shape the immersive world of Emotional Weather.

01 Artistic Vision

Nature-Inspired Aesthetics: The design draws from classical landscape paintings, using nature’s dynamic beauty to evoke emotional responses.
Seasonal and Atmospheric Shifts: Weather transitions such as rain, fog, and sunlight are rendered with shifting tones, light intensities, and volumetric effects.

Visual Language

Symbolic Weather Events: Weather patterns are integrated as narrative devices, symbolizing emotional states like melancholy (rain), tension (storms), and tranquility (clear skies).
Art Generation Through Environment: Every visual element contributes to storytelling by reflecting internal emotional journeys through changing environments.New art generate by real-time rendering.

Art-Driven Narrative Design

02 Conceptual Weather Selection

Emotional Symbolism: Selected weather types such as rain, fog, and storms were chosen for their symbolic connections to human emotions like sadness, uncertainty, and tension.
Visual Impact: Atmospheric conditions like fog and sunlight offer strong visual contrasts that reinforce narrative shifts and evoke emotional intensity.
Narrative Functionality: Weather transitions act as environmental storytelling tools, reflecting changes in emotional states through evolving landscapes.

Design Rationale for Weather Types

Selected ⬇️

Artistic Interpretation: While inspired by real-world weather patterns, artistic exaggeration ensures that each weather effect maximizes its emotional and narrative potential.
Dynamic Transitions: Seamless blending between weather conditions ensures fluid narrative progression without breaking immersion.

Balancing Realism and Abstraction

03 Environmental Composition

The environmental design in Emotional Weather creates immersive, evolving landscapes that reflect emotional and narrative shifts. By layering modular components and integrating dynamic environmental systems, the experience guides viewers through a visually and emotionally transformative world.

Layered Natural Scenes

Modular Environments: Landscapes are composed of modular elements like trees, rocks, and water features, forming dynamic and emotionally responsive worlds. Each element is intentionally designed to reflect shifting emotional states, ensuring that every environment evolves in harmony with the narrative.
Dynamic Day-Night Cycle: An evolving time-of-day system adjusts ambient lighting and skybox visuals, mirroring the passage of time and reinforcing the emotional flow of the experience. This creates an immersive cycle where light and shadow guide the viewer through changing emotional landscapes.

Emotional Progression

Emotional Milestones: The environmental experience is designed to lead viewers through emotional arcs, where weather-driven narrative shifts act as symbolic transitions in a personal emotional journey.
Exploration and Discovery: Open-ended environmental exploration encourages viewers to engage deeply, uncovering evolving narratives that mirror emotional transformations shaped by dynamic weather changes.

04 Technical Art Integration

The artistic and technical aspects of Emotional Weather are deeply intertwined, with every rendering technique chosen to heighten emotional engagement. By combining artistic intent with cutting-edge technology, the project transforms abstract emotions into vivid interactive experiences.

Artistic Rendering as a Digital Canvas

Real-Time Artistic Expression: The project envisions the environment as a responsive, evolving canvas where technology and art converge. Each weather change acts as a digital brushstroke, dynamically composing scenes that reflect emotional states through fluid, continuous visual transformations.
Art as Interactive Process: Emotional Weather transforms the viewer from passive observer to active participant, enabling exploration of changing environmental art through intuitive controls. This interactivity extends the creative process, making artistic interpretation collaborative and experiential.
Blending Artistic Vision and Technical Precision: The project harmonizes traditional artistic principles with advanced rendering technology, ensuring that technical processes like shader programming and volumetric lighting are not merely functional but integral to the visual narrative. The result is an ever-evolving artistic experience driven by both design intent and real-time computational power.

Art-Driven Rendering Techniques

Procedural Shaders: Custom shaders act as both technical tools and artistic mediums, generating effects like water reflections, fog density, and sunbeams breaking through clouds to evoke emotions like serenity, mystery, and awe.
Volumetric Lighting: Ray-marched clouds and volumetric fog enhance atmospheric depth, turning environmental changes into emotionally charged visual narratives.

Art-Integrated Performance Optimization

Real-Time Rendering Pipelines: Efficient asset streaming and level-of-detail (LOD) management balance artistic complexity with system performance, ensuring fluid and immersive visuals.
Optimized Particle Effects: The Niagara system powers dynamic weather-based particles like rain and snow, combining technical precision with artistic expression to create emotionally resonant environmental effects.

Production

Emotional Weather Project Contains Several Core Systems:

Landscape and Materials: Modular terrain elements, seamless textures, and environmental effects such as puddles and wind create a reactive, weather-driven environment.
Sky and Atmosphere: Dynamic skyboxes, volumetric clouds, and the day-night cycle simulate realistic atmospheric transitions and evoke emotional resonance.
Weather Systems: Niagara particle effects power dynamic rain and lightning, supported by liquid materials for a heightened sense of realism.
Control and UI: Real-time control mechanisms, render targets, data storage, and widget-based interfaces enable intuitive and seamless user interactions with the evolving environment.

01 Weather Control System Overview

This integrated system ensures real-time, responsive weather dynamics that combine technical precision with emotional storytelling by implementing the following modules.

User Input: Controls for weather (sunny, cloudy, rainy) and time (day, night).
Core Handlers: Process user inputs and update weather states in real-time.
Dynamic Elements:
- Clouds: Adjust color, density, and movement through the Cloud Manager.
- Terrain: Reflect weather changes via textures, puddles, and foliage.
- Rain & Thunder: Managed by the Niagara system for immersive effects.
- Lighting: Blends environment and landscape lighting for seamless transitions.

02 Day-Night Shift System

To ensure users can enjoy the scene of sunrise, sunset ,starry stars and other touching moment that could affect user emotion, a dynamic say-night shift system that allow users control the time in the system has been implemented.

03 Volumetric Cloud

Volumetric clouds serve as a crucial element in creating immersive and emotionally resonant environments in Emotional Weather. Their dynamic nature not only enhances realism but also acts as a symbolic bridge between the visual narrative and the user’s emotional experience

Ray-march Cloud Algorithm

Ray-marching, a technique used to calculate the path of light through volumetric data, allows for detailed and complex rendering of cloud formations, enabling dynamic density, lighting, and movement.

Ray-march Cloud Effect

A ray-marched volumetric cloud system generates realistic, ever-changing skies, ensuring that each moment feels alive and unique.
Through the refraction of light and clouds, spectacular scenes like the Tyndall effect can occur.
Cloud density, color, and movement adjust in real time, reflecting and reinforcing the evolving emotional tone of the environment.

04 Dynamic Terrain

The terrain system in Emotional Weather acts as the foundation of the interactive environment, seamlessly blending artistic intent with technical precision. By dynamically responding to weather conditions and enhancing visual storytelling, the terrain fosters a deep emotional connection between the user and the evolving landscape.

Puddles

A tangible representation of rain is the puddle, and by creating puddles on the terrain and then varying the size of the puddle, both rainy and rainy weather can be shaped more realistically.

Raining-Large Puddle

After Rain-Small Puddle

Sunny-No Puddle

Foliages

Wind is a very important part of weather, but wind cannot be seen directly. At first I planned to represent the wind in the perspective of rain, but then I decided to add Foliages to the project and make these Foliages blended by vertex shader to visualize the wind.

Prevent Material Duplication

Blended material gets the current weather information from the slider in the UI, and then decides which textures to blend (via the Lerp function for blending effects).
e.g. in sunny to cloudy weather, the material blends the sunny painting, the cloudy painting, and the real-time rendered scenery. This approach ensures that changes in the painting are not rigid and inspires more possibilities .

Select Texture based on Vector of terrain

According to the vector of the location of the terrain, area to determine whether the terrain is flat or slope, and then choose a different texture for it, and through the lerp way to make the slope and the intersection of flat texture natural transition

05 VFX: Rain and Atmospheric Effects

Rain

Real-Time Rain Simulation: The Niagara particle system powers rain, storms, and fog effects, creating dynamic and immersive weather scenes. To balance performance and functionality, CPU-based rain is used for foreground rendering to enable collision detection for interactive elements, while GPU-based rain handles distant and background effects to reduce computational cost.

GPU Rain in the background

CPU Rain in the foreground

Thunder

Thunder and lightning are common elements in stormy weather, and their appearance accompanied by loud noises can be a good way to mobilise the player's emotions.

A point Light lights up the scene when lightning

Screen Space Point Light

A advanced niagara system create random lightning effect

Lightning Niagara

wait for 1-3s ➡️

Play Thunder 1-3s after Lightning

Thunder Sound

In order to better present the effect of thunder and lightning, the thunder and lightning particle system underwent several iterations. The initial system simply included a lightning bolt but no other effects, so a point light source was added to light up the whole scene for further simulation. In addition to this, there is a time lag between the lightning and the thunder in order to make the effect more realistic.

06 Art and RealTime Render

Apart from build a emotional connection with users, the project also reflect on using code as a tool to create art work, with this mission, the project featured a real-time rendered art work on the canvas in the scene. And it has be blended with some fine artwork.

Weather in classic painting

Weather in Rendered Scene

Newly Generated Art Work

➡️

Blended Material

Blended material gets the current weather information from the slider in the UI, and then decides which textures to blend (via the Lerp function for blending effects), e.g. in sunny to cloudy weather, the material blends the sunny painting, the cloudy painting, and the real-time rendered scenery. This approach ensures that changes in the painting are not rigid and inspires more possibilities .

07 Data Assets, UI and Sound Design

Data Assets

Weather System Data Assets- Take Terrain Puddle as An Example

The project controls presets through data Asset and Enum. It provides a silky smooth translation between weather by using user inputs to X-axis values and setting individual elements of the weather system to Y-axis values.

User Interface

Finally, the project consists of two sliders: a time-control slider and a weather-control slider, both triggered by the on-value-changed method, with additional logic set to disable the time-control slider on rainy days.

The time-control slider implements the Day-Night shift. the weather-control slider implements the weather-control slider.
The main functionality of the weather control slider is to control the sunny-cloudy-rainy day shift and the generation of puddles on the ground and trigger the associated particle system.

Sound Design

To Help the player immerse themselves and build an emotion connection in the virtual weather system by trying to keep things simple in terms of sound effects and removing unnecessary sound effects like background music and wind sound effect.

Sunny

Rainy

Storm

Ambient Sound

Thunder Sound

BGM

WindSound

Other Sound

Iteration

Color Correction

Compare with real-life weather scenery,
adjust colour presets for different weather through videos/pictures.

Screen Raindrop Layer

To give the project a better rain effect, I added a glass rain effect to the rainy scene, so the viewer would see raindrops dripping down onto the lens and changing as the rain got heavier.

Light &Environment Mixer

By further changing the settings in Light and Environment Mixer, the scene’s visual effect has been improved and the ambient has become more vivid.

Further Development

Artistic integration
The project tries to integrate real-time rendering technology with art, but the current integration is through algorithms for overlaying and blending, a better solution is to analyse the paintings of different painters to create different material function and style presets, and then use them to create new art work

Further Extension
The research questions could have been better addressed with the inclusion of an anthropological study. For instance, participants could have used the simulation system while their emotions were recorded using a participant observation approach. The exploration of integrating art and computer graphics technology through fusion rendering of classic paintings and real-time weather could have been enhanced by creating more weather variations and offering players the option to paint more.