A groundbreaking digital preservation challenge has created probably the most complete 3D map of Alcatraz Island ever produced, combining cutting-edge drone expertise, synthetic intelligence, and actuality seize programs to doc one in every of America’s most infamous historic websites. Pete Kelsey, founding father of VCTO Labs, offered the exceptional achievements of this formidable challenge at Intergeo 2025 in Frankfurt, Germany, the place he detailed how his worldwide staff of over 20 volunteers accomplished what represents one of many largest-scale digital twin initiatives of an American landmark.
The challenge, accomplished in December 2023 in partnership with the Nationwide Park Service, represents a brand new frontier in cultural heritage preservation. Throughout his presentation, Kelsey mentioned the technical challenges and groundbreaking conclusions from what he described as one of the vital complete conservation initiatives ever carried out in america. Rising sea ranges, seismic exercise from the San Andreas Fault, and the influence of over a million annual guests have created urgent considerations for the 22.5-acre island’s structural integrity.
Residing and Working in America’s Most Well-known Jail
Kelsey’s staff spent three weeks residing within the precise jail cells on Alcatraz, working across the clock to seize 4 terabytes of information utilizing terrestrial laser scanning, drone-based LiDAR, multispectral scanning, photogrammetry, and robotic programs. The weird residing preparations turned a part of the challenge’s distinctive character.
“I used to be in D-block, third tier, cell quantity 31. House candy house!” joked Kelsey throughout his presentation. Regardless of considerations in regards to the island’s reported paranormal exercise, Kelsey mentioned his sleep was glorious, although he needed to relocate as soon as when a retired police officer on the staff’s loud night breathing turned disruptive. His unnamed colleague’s momentary transfer to the mugshot room below the previous hospital proved short-lived on account of unexplained sounds from upstairs, together with what gave the impression of a piano participant and crowds of 50-100 individuals.
The cruel working situations prolonged past ghostly encounters. The staff needed to deliver all their very own provides, together with sleeping luggage and meals, whereas sharing a single lavatory and brushing their enamel at ingesting fountains. Working primarily in the course of the early morning hours earlier than vacationers arrived, the staff had solely two to three-hour home windows to seize information with out disrupting the customer expertise.
Superior Expertise Arsenal
The challenge’s success relied on an unprecedented mixture of actuality seize applied sciences. With a background in development and surveying, Kelsey emphasised the significance of building a complicated survey management community that achieved 6mm precision. This baseline made the in any other case almost unimaginable activity of mixing and reconciling information from completely different sources manageable, making a basis from which the staff might work.
“Beginning with a survey management community helped lots – there isn’t a endgame with out that,” Kelsey defined.
Drone Expertise at Work
The challenge’s aerial information assortment represented one of the vital refined multi-platform drone operations ever carried out at a Nationwide Park Service website, the place flights are normally banned. The staff deployed a number of specialised unmanned aerial automobiles, every chosen for particular mission necessities throughout Alcatraz’s numerous terrain and structural challenges.
The Impressed Flight 1200i served as the first heavy-lift platform, able to carrying bigger sensor payloads for prolonged flight instances across the island’s perimeter and elevated buildings. For precision mapping of the island’s complicated topography, the staff utilized the EagleNXT eBee fixed-wing drone, which supplied environment friendly protection of enormous areas whereas sustaining the accuracy required for survey-grade documentation.
Maybe most crucial to the challenge’s success was the deployment of the Elios 3 collision-resistant drone for confined inside areas. This specialised platform allowed the staff to securely navigate the slim corridors, cells, and utility areas inside the jail buildings the place conventional drones would have been unimaginable to function. The Elios 3’s protecting cage and superior collision-avoidance programs enabled detailed scanning of inaccessible areas that had by no means been comprehensively documented.
ESRI performed an important position within the Alcatraz challenge, offering each photogrammetry experience and superior processing capabilities by their ArcGIS Actuality suite of software program merchandise. The photogrammetry element represented one in every of three crucial aerial information assortment flights, alongside the LiDAR and multispectral scanning operations. ESRI’s contribution was important for creating the photorealistic texture mapping that might later be built-in with the LiDAR level cloud information to provide the ultimate digital twin.
Phoenix LiDAR Techniques supplied the RANGER-U120 high-precision, long-range LiDAR scanner, which turned the cornerstone of the aerial mapping operations. This survey-grade system delivered the centimeter-level accuracy required for creating baseline documentation that might assist a long time of future monitoring. The RANGER-U120’s capability to penetrate vegetation and seize exact measurements of deteriorating concrete and metal buildings proved important for documenting the island’s present structural situation.
The aerial operations required cautious coordination with Federal Aviation Administration rules and Nationwide Park Service necessities, significantly given Alcatraz’s location in restricted San Francisco Bay airspace. Flight operations had been additional sophisticated by the island’s seabird nesting seasons, which restricted entry to sure areas throughout crucial breeding durations. The staff needed to work inside slim time home windows, usually throughout early morning hours earlier than vacationer operations started, maximizing their transient alternatives to gather complete aerial information with out disrupting customer experiences.
For areas with environmental hazards containing lead and asbestos, (the place the usage of drones risked spreading poisonous mud) the staff deployed a Boston Dynamics robotic canine nicknamed “Spot”, geared up with Emesent’s Hovermap LiDAR system. This robotic resolution prevented human publicity to poisonous supplies whereas sustaining information assortment capabilities. The Hovermap system might seize steady scans at Spot’s most strolling tempo, making it considerably sooner than typical terrestrial laser scanning strategies.
RIEGL USA contributed terrestrial laser scanning gear, together with the VZ-600i for inside scans and the VZ-2000i for long-range exterior scanning able to reaching 2.5 kilometers distance, capturing parts of the San Francisco shoreline.
Processing Energy Challenges
One of many challenge’s most important technical challenges got here from Alcatraz’s restricted web connectivity, which dominated out cloud-based information processing. The answer got here from AMD, which supplied a prototype workstation geared up with a 96-core Threadripper PRO processor and half a terabyte of RAM.
“The workplace on Alcatraz was actually chilly, however that machine saved us heat,” Kelsey famous.
Epic Video games Integration and Digital Actuality Functions
Epic Video games performed a major position in creating the ultimate digital twin expertise utilizing their RealityScan, Twinmotion, and Unreal Engine applied sciences. The RealityScan 2.0 software program was used to generate an in depth digital duplicate from aerial LiDAR, images, and Bentley’s Cesium tiles, whereas Unreal Engine supplied the framework for creating immersive, interactive experiences.
The Epic Video games integration allows a number of functions past easy documentation. In future, the mannequin might assist digital tourism capabilities, permitting guests to discover areas usually restricted to the general public. “What about all of the individuals who wish to go behind the velvet ropes? With a pair of goggles, I can present that world to you,” Kelsey defined.
Historic Discoveries and Documentation
The excellent scanning revealed beforehand unknown historic options and documented well-known parts of Alcatraz’s previous. The staff efficiently mapped the whole escape route used within the notorious 1962 breakout by Frank Morris and brothers Clarence and John Anglin. Their scans captured the upstairs workshop the place the escapees constructed their rafts and the routes they used to succeed in the water.
Multi-spectral scanning strategies revealed the stays of a helipad added in the course of the Nineteen Sixties, uncovering island historical past that even Nationwide Park Service workers had forgotten. The staff additionally documented graffiti all through the power, together with notes written by guards and pictures created by rioting inmates, although Kelsey famous wryly that some prisoner paintings was too specific for public show.
Maybe most importantly, the challenge revealed the unique Civil Conflict-era sallyport, or secured entrance, on the japanese aspect of the island – a function that had not been seen for over a century. This discovery supplied new insights into Alcatraz’s navy historical past earlier than its well-known jail years.
Conservation and Local weather Change Functions
The digital twin serves a number of conservation functions, offering baseline documentation to observe the results of local weather change, seismic exercise, and customer influence on deteriorating buildings. The Nationwide Park Service can now examine future surveys towards this complete dataset to trace adjustments over time.
“On the very least, we’ve created a baseline information set that future surveys may be primarily based on. Whether or not it’s sea degree rise or erosion, or San Francisco’s seismic exercise, we’ll have the ability to observe the adjustments and be part of the way forward for the island,” Kelsey defined.
Future Income and Academic Functions of Digital Twins
Past conservation, the digital twin might create new income alternatives for the Nationwide Park Service by digital tourism and academic functions. The expertise allows immersive experiences for individuals unable to go to the island bodily and gives academic establishments with unprecedented entry to historic websites.
Kelsey envisions the challenge contributing to his “BHAG” (Large Bushy Audacious Aim): creating the world’s first digital nationwide park. The filming and digital actuality expertise might turn out to be a major income driver for the park service whereas increasing entry to historic schooling.
“Actuality seize is educating the shopper in regards to the asset they now have, and the way it might be used for something, from upkeep to licensing,” Kelsey famous. The excellent documentation additionally helps upkeep planning by offering detailed structural data for preservation specialists.
Technical Classes and Business Affect
The Alcatraz challenge demonstrated a number of key rules for large-scale actuality seize initiatives. Kelsey emphasised the significance of thorough planning, noting that 9 months of preparation preceded the three-week information assortment part. Acquiring permits for drone operations in a nationwide park and coordinating round seabird nesting seasons required in depth advance work.
The challenge’s success depended closely on collaboration between a number of expertise corporations and volunteers who contributed gear and experience. Phoenix LiDAR Techniques, Emesent, RIEGL USA, Boston Dynamics, AMD, Epic Video games, and quite a few different companions supplied important parts of the great resolution.
For drone professionals planning related initiatives, Kelsey beneficial beginning with a exact survey management community and calling on skilled companions. “Having executed giant strategic initiatives like this earlier than, I had a way of learn how to do it. I relied on individuals lots smarter than me – I known as in each favor anybody ever owed me,” he defined.
Advancing Digital Preservation Requirements
The Alcatraz digital twin challenge units new requirements for cultural heritage preservation, demonstrating how superior geospatial applied sciences can defend America’s historic landmarks for future generations. As rising sea ranges and excessive climate occasions more and more threaten coastal historic websites, Kelsey’s work gives a replicable mannequin for complete digital documentation.
“We make the invisible seen. We make individuals see views and factors of view which have by no means been seen earlier than,” Kelsey mentioned, referring to actuality seize professionals. The challenge’s mixture of a number of applied sciences, robotic programs, and superior processing capabilities creates a template for preserving different vital historic websites dealing with related environmental and structural challenges.
The excellent dataset now serves as a everlasting document of Alcatraz Island’s situation in 2023, making certain that future researchers and conservationists could have exact documentation of this iconic American landmark no matter how local weather change and time could alter its bodily construction.
Extra data on VCTO Labs and Pete’s different initiatives can be found right here.
Learn Extra
Ian McNabb is a journalist specializing in drone expertise and life-style content material at Dronelife. He’s primarily based between Boston and NH and, when not writing, enjoys mountaineering and Boston space sports activities.
