Unreal 4 VR C++ – Light Painter

A VR prototype for painting and writing in 3D space built in UE4 C++. The player holds a paint brush in one hand and a palette menu in the other. Pressing the trigger starts a stroke; releasing ends it. Paintings persist across sessions and can be opened from a main menu gallery.

An original project — not based on a course.

📺 Demo Video | 🔗 GitHub Repository

Stroke rendering: instanced static meshes

The most important technical decision in the project is how strokes are rendered. Each stroke is a single AActor with two UInstancedStaticMeshComponent arrays:

SegmentMeshes — one instance per pair of consecutive control points, rendered as a cylinder scaled along X to fill the gap exactly.
JointMeshes — one instance per control point, rendered as a sphere that hides the seam between consecutive segments.

The alternative — spawning a separate AActor per segment — would produce thousands of actors and draw calls for a painting with many strokes. Instanced meshes collapse all segments of a stroke into a single draw call per material regardless of stroke length. Rendering cost stays flat as the painting grows.

Each tick while the trigger is held, APaintBrushHandController calls AStroke::Update() with the controller’s world location. Update() appends the location to ControlPointLocations, then builds the segment transform:

cpp

			
// X scale = segment length so the mesh stretches to fill the gap exactly.
FVector Scale = FVector(Segment.Size(), 1.0f, 1.0f);
// Rotate so the mesh +X axis aligns with the stroke direction.
FQuat Rotation = FQuat::FindBetweenNormals(FVector::ForwardVector, SegmentNormal);
// Convert to local space — instanced mesh instances are relative to the actor.
FVector Location = GetTransform().InverseTransformPosition(CursorWorldLocation);

		

The local-space conversion is essential. Instanced mesh instances are stored relative to the actor’s transform, not in world space. Without it, strokes drawn anywhere except the world origin would be mispositioned.

The first Update() call only adds a joint — there is no previous point to form a segment. Zero-length segments (controller held still) are skipped explicitly to avoid degenerate mesh transforms with zero X scale.

Save/Load: control point replay

No mesh data is stored. FStrokeState contains only two fields:

cpp

			
USTRUCT()
struct FStrokeState
{
    UPROPERTY() TSubclassOf<AStroke> StrokeClass;
    UPROPERTY() TArray<FVector> ControlPointLocations;
};

		

On save, UPainterSaveGame::SerializeFromWorld() iterates all AStroke actors and calls SerializeToStrokeState() on each — storing the class and the control point array.

On load, DeserializeToWorld() destroys all existing stroke actors, then for each saved FStrokeState spawns a new AStroke of the correct class and replays all saved control points through AStroke::Update(). The geometry reconstructs deterministically — the same sequence of Update() calls always produces the same instanced mesh instances.

This approach keeps save files small (just FVector arrays), makes them independent of mesh asset changes, and means the serialization logic is exactly one copy of the painting loop.

The GUID slot system and its index

Each painting gets a unique GUID as its save slot name, generated at creation time:

cpp

PainterSaveGame->SlotName = FGuid::NewGuid().ToString();

The GUID is the primary key — UGameplayStatics::SaveGameToSlot() and LoadGameFromSlot() use it directly. This means paintings never collide even if two are created at the same instant.

The problem: how does the main menu know which GUIDs exist? UGameplayStatics::LoadGameFromSlot() requires you to know the slot name in advance — there is no “list all slots” API in UE4.

The solution is UPainterSaveGameIndex, a second USaveGame stored under the fixed slot name "PaintingIndex". It holds a TArray<FString> of all painting GUIDs. When a painting is created, its GUID is appended to the index and the index is re-saved. The main menu loads "PaintingIndex" first, then iterates the GUID list to populate the gallery.

UPainterSaveGameIndex::Load() creates and saves a new empty index on first run rather than returning null — so all call sites can assume a valid index is always returned.

SlotName via travel URL options

When the player clicks a painting card in the main menu, UPaintingGridCard::OnCardButtonClicked() opens the painting level:

cpp

			
const FString Options = TEXT("SlotName=") + SlotName->GetText().ToString();
UGameplayStatics::OpenLevel(GetWorld(), TEXT("Canvas"), true, Options);

On the other end, APaintingGameMode::InitGame() extracts it:

cpp

SlotName = UGameplayStatics::ParseOption(Options, TEXT("SlotName"));

InitGame() is critical here — it fires before BeginPlay(), so Load() has the slot name available when it runs in BeginPlay(). Using a member variable set in a Blueprint event or a game instance would introduce timing dependencies or global state. The URL option is self-contained: the slot name travels with the level transition.

Two interaction models for VR UI

The project uses two different interaction systems for two different UI contexts.

Laser pointer — The right hand controller (AUIPointerHandController) carries a UWidgetInteractionComponent that raycasts against world-space UWidgetComponent panels. OnTriggerPress() calls PressPointerKey(LeftMouseButton); release calls ReleasePointerKey. This is appropriate for the main menu gallery, which is a panel placed in front of the player at a distance — the natural interaction is pointing and clicking.

Physical touch — The palette menu is a UWidgetComponent mounted on the left hand controller (APaletteMenuHandController). The right hand controller tip carries a UWidgetTouchingComponent — a UWidgetInteractionComponent subclass that overrides TickComponent():

cpp

			
if (IsOverInteractableWidget() && !bIsClicked)
{
    PressPointerKey(EKeys::LeftMouseButton);
    bIsClicked = true;
}
if (!IsOverInteractableWidget() && bIsClicked)
{
    ReleasePointerKey(EKeys::LeftMouseButton);
    bIsClicked = false;
}

		

No trigger is required — hovering over a widget element is sufficient to activate it. A bIsClicked flag prevents repeated press events while the controller remains over the same element. This is appropriate for a hand-mounted menu: the player physically moves their right hand onto the palette surface to select options, which is more ergonomic than aiming a laser at their own left hand.

Thumbstick pagination

AVRPawn::PaginateRightAxisInput() receives a continuous axis value (-1.0 to 1.0) from the left thumbstick. It converts this to a discrete +1 / 0 / -1 direction and fires UpdateCurrentPage() only when the direction changes AND is non-zero:

cpp

			
int32 PageIndexOffset = 0;
PageIndexOffset += AxisValue >  PaginationThumbstickThreshold ?  1 : 0;
PageIndexOffset += AxisValue < -PaginationThumbstickThreshold ? -1 : 0;
if (PageIndexOffset != LastPageIndexOffset && PageIndexOffset != 0)
{
    UpdateCurrentPage(PageIndexOffset);
}
LastPageIndexOffset = PageIndexOffset;

		

The LastPageIndexOffset != PageIndexOffset check detects a direction change. The PageIndexOffset != 0 check prevents firing when the thumbstick returns to center. Together they give one page event per directional flick regardless of how long the thumbstick is held past the threshold.

Engine version: Unreal Engine 4