Machine perception / visual computing

Neural Renderer

A scene-reconstruction system that turns ordinary photographs into navigable, continuously rendered three-dimensional environments.

PyTorchCUDANeRFThree.js
IP / NEURAL
Statusactive
Year2026
Training7 min / scene
Rendering60 FPS
RepositoryPrivate beta

The problem

Turn ordinary photographs into a navigable 3D scene without a specialist capture setup.

My contribution

Connected image calibration, neural reconstruction, and an interactive viewer into one experimental workflow.

Results so far

The prototype supports interactive scene exploration; capture guidance and validation remain in progress.

The full story

Why build another renderer?

Neural radiance fields are often presented as an impressive output image and a compact equation. The difficult engineering lies between those two things: unreliable camera poses, memory pressure, long feedback loops, and a representation that does not naturally fit the web.

This project treats reconstruction as a complete product pipeline rather than a model demo. A scene begins as an unstructured set of photographs and ends as a progressively streamed experience that can be opened on an ordinary device.

The representation

The scene is a continuous volumetric function:

FΘ:(x,d)↦(c,σ)F_\Theta : (x, d) \mapsto (c, \sigma)

Position xx and direction dd produce colour cc and density σ\sigma. Rendering integrates those predictions along a camera ray. The current implementation replaces a deep positional network with a compact multiresolution hash grid, spending capacity where the scene contains detail.

What matters now

Raw render quality is no longer the bottleneck. The active work is about interaction: making partial downloads useful, exposing uncertainty, and helping a person understand where a reconstruction is trustworthy. That is the difference between a paper implementation and a tool.

Timeline

Progress,
without revisionism.

A chronological record of milestones, course corrections, and the next concrete step.

ResearchPrototypeValidationRelease
  1. 01

    Research baseline

    Reproduced the original NeRF paper and established quality and speed benchmarks.

    complete
  2. 02

    Instant encoding

    Introduced a multiresolution hash grid and reduced training from hours to minutes.

    complete
  3. 03

    Custom ray marcher

    Moved sampling and compositing into a fused CUDA kernel.

    complete
  4. 04

    Browser editor

    The current focus: a progressive Three.js viewer with authoring tools.

    active
  5. 05

    SDK release

    Package the renderer and streaming protocol behind a stable public API.

    planned