Machine perception / visual computing

Neural Renderer

A scene-reconstruction system that turns ordinary photographs into navigable, continuously rendered three-dimensional environments.

PyTorchCUDANeRFThree.js
IP / NEURAL
Statusactive
Year2026
Training7 min / scene
Rendering60 FPS
RepositoryPrivate beta
ResearchPrototypeValidationRelease

System anatomy

How the project
fits together.

This map is driven directly by the project data in Keystatic. Every layer groups the decisions, components, and current state of the work.

01

Acquisition

Turn an imperfect photo set into calibrated observations.

complete

Camera calibration

Intrinsic and extrinsic parameters recovered through robust feature matching.

active

Capture guidance

Live quality feedback flags coverage gaps before training begins.

02

Representation

Encode geometry and appearance into a compact continuous field.

complete

Hash-grid encoder

Multi-resolution spatial features replace the original positional encoding.

planned

Dynamic objects

A temporal branch will separate moving subjects from static geometry.

03

Synthesis

Transform the learned field into pixels at interactive speed.

complete

CUDA ray marcher

Empty-space skipping and fused sampling reduce render cost.

active

Web streaming

Progressive field delivery makes scenes explorable before full download.

04

Experience

Make a research artifact feel like a useful creative tool.

active

Scene editor

Camera paths, crop volumes, and exposure can be edited in-browser.

planned

Public SDK

A small API will embed trained scenes into third-party products.

The full story

Why build another renderer?

Neural radiance fields are often presented as an impressive output image and a compact equation. The difficult engineering lies between those two things: unreliable camera poses, memory pressure, long feedback loops, and a representation that does not naturally fit the web.

This project treats reconstruction as a complete product pipeline rather than a model demo. A scene begins as an unstructured set of photographs and ends as a progressively streamed experience that can be opened on an ordinary device.

The representation

The scene is a continuous volumetric function:

FΘ:(x,d)(c,σ)F_\Theta : (x, d) \mapsto (c, \sigma)

Position xx and direction dd produce colour cc and density σ\sigma. Rendering integrates those predictions along a camera ray. The current implementation replaces a deep positional network with a compact multiresolution hash grid, spending capacity where the scene contains detail.

What matters now

Raw render quality is no longer the bottleneck. The active work is about interaction: making partial downloads useful, exposing uncertainty, and helping a person understand where a reconstruction is trustworthy. That is the difference between a paper implementation and a tool.

Timeline

Progress,
without revisionism.

A chronological record of milestones, course corrections, and the next concrete step.

  1. 01

    Research baseline

    Reproduced the original NeRF paper and established quality and speed benchmarks.

    complete
  2. 02

    Instant encoding

    Introduced a multiresolution hash grid and reduced training from hours to minutes.

    complete
  3. 03

    Custom ray marcher

    Moved sampling and compositing into a fused CUDA kernel.

    complete
  4. 04

    Browser editor

    The current focus: a progressive Three.js viewer with authoring tools.

    active
  5. 05

    SDK release

    Package the renderer and streaming protocol behind a stable public API.

    planned