Posts under Spatial Computing topic

Post

Replies

Boosts

Views

Activity

visionOS 27.0 simulator restarts (backboardd EXC_BREAKPOINT in DRValidateIndices) on MeshResource.generate(from:) for a valid multi-part mesh
Since moving to Xcode 27.0 (27A266a) with the visionOS 27.0 simulator runtime (24M362), generating a mesh with several parts through MeshResource.generate(from: MeshResource.Contents) makes the simulator go black for a couple of seconds and restart. The app does not crash; the simulator's render server does: Thread ... Crashed :: com.apple.CoreRE.ResourceFetchManager-service-queue 0 DirectResource specialized static IndexValidation._validate_neon(_:vertexCount:) 1 DirectResource DRValidateIndices 2 CoreRE re::(anonymous namespace)::validatePrivateData(id<REMeshPayload>, unsigned char const*, unsigned long) 3 CoreRE -[REMultiBufferMeshPayload initWithCoder:] 4 Foundation _decodeObject Environment: MacBook Pro (M1 Pro, 16 GB), macOS 27.0 (26A428), Xcode 27.0 (27A266a), visionOS 27.0 SDK (24M361); failing runtime visionOS 27.0 simulator (24M362); working: the visionOS 26.5 simulator runtime (23O470) under the same Xcode, and an Apple Vision Pro on visionOS 27. What I have established: It reproduces with a fully synthetic mesh: one model, three parts (1330 / 20 / 24 vertices, 9513 / 72 / 120 indices), random positions, normals, UVs and triangle indices. Only the part layout matters. The mesh is valid: every part's indices are within its own vertex count, counts are multiples of three, attribute buffers match. RealityKit's app-side validator agrees and lets it through (it does reject a broken part with REMeshPartDescriptor: index buffer payload range is invalid), then the render server's validator traps on the accepted payload. Single-part meshes of any size, primitives, and compiled .reality scenes are fine. Same mesh, same Xcode: fine on a simulator device created on the 26.5 runtime, fine on a visionOS 27 device. Only the 27.0 simulator runtime traps. FB number: FB24831983. Workarounds so far: a simulator device on the 26.5 runtime, or generating one MeshResource per part. Has anyone seen DRValidateIndices trapping on the 27.0 simulator?
1
0
615
2h
Logitech Muse never appears under Settings > General > Spatial Accessory — spatial tracking permission never granted
I'm building a visionOS 26 app that uses SpatialTrackingSession with .accessory tracking and AnchoringComponent.AccessoryAnchoringSource to track a Logitech Muse, following the pattern from the WWDC25 "Explore spatial accessory input on visionOS" session (and matching two independently published implementations I found). What works: The Muse pairs fine over Bluetooth, connects as a GCStylus (productCategory == .spatialStylus), GCStylusDidConnect fires, and button input (pressedDidChangeHandler) works correctly. What doesn't: SpatialTrackingSession.run() always reports .accessory as unavailable, so AnchoringComponent.AccessoryAnchoringSource(device:) never succeeds. Checking Settings > General > Spatial Accessory on the device shows the Muse isn't listed there at all — not even a battery/status entry. Settings > Privacy & Security > Spatial Accessory also never lists our app. What I've tried: Forgetting the Muse in Bluetooth and re-pairing Full power-cycle of both Bluetooth and the Muse itself Confirmed NSAccessoryTrackingUsageDescription is set and the Accessory Tracking + Game Controllers (Spatial Gamepad) capabilities are enabled in Xcode Confirmed via Apple's own docs, the WWDC session, and two developer write-ups that no special commissioning step beyond standard Bluetooth pairing should be needed For comparison, XR Putt (which also uses the Muse) works fine and does show up with a green checkmark under Spatial Accessory permissions on the same headset. Has anyone hit this specific "device never gets commissioned as a spatial accessory" issue, or knows what actually triggers that commissioning beyond generic Bluetooth pairing? Happy to share more logs/details.
1
0
702
1d
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
7
8
3.8k
1d
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
7
1
1.5k
1d
WebXR Augmented Reality Module
On visionOS 26.5, I noticed the Safari Extension 'WebXR Augmented Reality Module' option. I've enabled this option in Safari Settings, and restarted Safari, but when I try to launch a WebXR experience that supports immersive-ar as option (for example this one: https://www.viverse.com/dPASDhP I don't see the option to Play in AR (like I do with a Quest 3S headset) Does anyone have any examples of websites that can make use of this new WebXR Augmented Reality Module feature flag in Safari, on visionOS?
3
4
1.6k
3d
Main Camera Error
An incomprehensible error occurs whenever I try to access Main Camera features. I have already acquired the Main Camera entitlement and granted the required permissions, yet the error persists. To make matters worse, even Apple’s provided sample code triggers the exact same issue. The error message is far too uninformative to diagnose the root cause, and I urgently need a resolution as this is completely blocking progress. Multiline [C:1-2] Error received: Invalidated by remote connection. failed to observe with mask <BLSXPCBacklightProxyObserverMask: 0x108accf80; didUpdateToState: YES; eventsArray: YES> error:<XPC error received on message reply handler (3:BSServiceConnectionErrorDomain) "The operation couldn’t be completed. XPC error received on message reply handler"> non-launching port is incompatible with service identifier "com.apple.PointerUI.pointeruid.default-service" nw_socket_copy_info [C1:2] getsockopt TCP_INFO failed [102: Operation not supported on socket] nw_socket_copy_info getsockopt TCP_INFO failed [102: Operation not supported on socket] Unable to find a compatible slice for binary archive at /System/Library/Frameworks/CoreImage.framework/ci_uberwrapper_bin.metallib *** Assertion failure in void ar_session_request_authorization(ar_session_t _Nonnull, ar_authorization_type_t, __strong ar_authorization_results_handler_t _Nonnull)_block_invoke(), session.m:1491, reason: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. <NSXPCConnection: 0x10e68dcc0> connection to service with pid 86 named com.apple.arkit.service.authorization: Exception caught during invocation of reply block to message 'requestAuthorizations:withReply:'. Exception: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. ( 0 CoreFoundation 0x0000000198b756d4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205972 1 libobjc.A.dylib 0x000000018d3a71cc objc_exception_throw + 88 2 CoreFoundation 0x0000000198b755c0 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205696 3 ARKitCore 0x00000002bbd640cc 53FDDDD4-8DD1-3E86-ACB0-8867D29AF781 + 479436 4 CoreFoundation 0x0000000198b7c5b4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1234356 5 CoreFoundation 0x0000000198b79558 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1221976 6 Foundation 0x000000019de4220c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2109964 7 Foundation 0x000000019de41dd0 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2108880 8 Foundation 0x000000019de4651c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2127132 9 libxpc.dylib 0x00000002c43d5a00 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 150016 10 libxpc.dylib 0x00000002c43c5678 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 83576 11 libdispatch.dylib 0x00000002c41045fc 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 108028 12 libdispatch.dylib 0x00000002c41083ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 123884 13 libdispatch.dylib 0x00000002c40f37c0 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 38848 14 libdispatch.dylib 0x00000002c40f4450 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 42064 15 libdispatch.dylib 0x00000002c40fe3ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 82924 16 libdispatch.dylib 0x00000002c40fdd18 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 81176 17 libsystem_pthread.dylib 0x00000002c4364e34 _pthread_wqthread + 292 18 libsystem_pthread.dylib 0x00000002c4362910 start_wqthread + 8 ) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) libc++abi: terminating due to uncaught exception of type NSException Xcode has killed the LLDB RPC server (pid: 2210) to allow the debugger to detach from your process. BlockQuote visionOS 27 beta 5
5
0
1k
4d
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
1
0
648
5d
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
4
0
1.8k
6d
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
5
0
1.3k
6d
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
1
0
115
6d
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
1
0
786
1w
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
1
0
1.2k
1w
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
3
0
3.2k
1w
Questions about RoomPlan, Room API, USDZ/STEP comparison, and extended spatial scanning
Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina
4
0
866
1w
Is there any technical documentation available for the iPhone’s LiDAR scanner?
Hi, I’m looking to develop an app that uses the iPhone’s LiDAR scanner and am trying to decide which model to buy. Is there any relevant technical documentation available on its capabilities, such as accuracy, resolution and range? I’d also like to understand whether there have been any improvements between generations, particularly with the iPhone 18 Pro. Any documentation or pointers in the right direction would be appreciated. Thanks!
0
0
279
1w
Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
0
0
501
1w
Generic spatial accessory receives LED reports but never produces an AccessoryAnchor
We are developing a generic spatial accessory for Apple Vision Pro using the “Working with generic spatial accessories” sample and a trained .referenceaccessory file. Environment: visionOS: [version/build] Xcode: [version/build] Hardware: Apple Vision Pro + nRF52840-based BLE accessory IMU: 6-axis IMU, 400 Hz HID reports LED constellation: 8 x 850 nm IR LEDs, driven through MOSFETs The BLE/HID and timing path appears to be working: The accessory connects successfully over BLE. ASA/HID enumeration succeeds. The firmware receives continuous LED Output Reports from visionOS. The Accessory Tracking Clock reaches READY. PWM scheduling reports no late events and no queue overflow. In the Accessory Tracking Debug View, multiple independent IR LED blobs are visible and stable at approximately 0.7 m. At least 4 LEDs are visible from normal viewing angles. The app resolves the .referenceaccessory and starts AccessoryTrackingProvider. However, ARKit never produces an AccessoryAnchor. We also have no diagnostic signal indicating whether the failure occurs at: LED candidate-point detection, constellation matching against the .referenceaccessory, or IMU/optical fusion. Questions: Is there a supported diagnostic API, log category, or Debug View indicator that distinguishes candidate-point rejection from constellation-match or fusion rejection? Are there known acquisition prerequisites in the current visionOS beta beyond the published Accessory Design Guidelines? Can Apple recommend a minimal validation procedure using the generic spatial accessory sample to confirm that the LED constellation is accepted by the tracking system? Is this expected to work in a volumetric window, or are there additional lifecycle/session requirements that can prevent initial acquisition? I can provide sanitized HID LED-report logs, timing data, Debug View captures, and a minimal reproducible project if useful.
10
0
2.8k
1w
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
2
0
1.1k
2w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
3
0
2.4k
2w
Anchoring Invalid
Hello. I trained an ObjectTracker.referenceobject using ML, and am using it in RCP. After adding Anchoring to the entity, I set the Target mode to Object and applied the trained model data ObjectTracker.referenceobject. However, there is no tracking effect after building for VisionPro. Are any permissions required? What should I do?
3
0
2.5k
2w
visionOS 27.0 simulator restarts (backboardd EXC_BREAKPOINT in DRValidateIndices) on MeshResource.generate(from:) for a valid multi-part mesh
Since moving to Xcode 27.0 (27A266a) with the visionOS 27.0 simulator runtime (24M362), generating a mesh with several parts through MeshResource.generate(from: MeshResource.Contents) makes the simulator go black for a couple of seconds and restart. The app does not crash; the simulator's render server does: Thread ... Crashed :: com.apple.CoreRE.ResourceFetchManager-service-queue 0 DirectResource specialized static IndexValidation._validate_neon(_:vertexCount:) 1 DirectResource DRValidateIndices 2 CoreRE re::(anonymous namespace)::validatePrivateData(id<REMeshPayload>, unsigned char const*, unsigned long) 3 CoreRE -[REMultiBufferMeshPayload initWithCoder:] 4 Foundation _decodeObject Environment: MacBook Pro (M1 Pro, 16 GB), macOS 27.0 (26A428), Xcode 27.0 (27A266a), visionOS 27.0 SDK (24M361); failing runtime visionOS 27.0 simulator (24M362); working: the visionOS 26.5 simulator runtime (23O470) under the same Xcode, and an Apple Vision Pro on visionOS 27. What I have established: It reproduces with a fully synthetic mesh: one model, three parts (1330 / 20 / 24 vertices, 9513 / 72 / 120 indices), random positions, normals, UVs and triangle indices. Only the part layout matters. The mesh is valid: every part's indices are within its own vertex count, counts are multiples of three, attribute buffers match. RealityKit's app-side validator agrees and lets it through (it does reject a broken part with REMeshPartDescriptor: index buffer payload range is invalid), then the render server's validator traps on the accepted payload. Single-part meshes of any size, primitives, and compiled .reality scenes are fine. Same mesh, same Xcode: fine on a simulator device created on the 26.5 runtime, fine on a visionOS 27 device. Only the 27.0 simulator runtime traps. FB number: FB24831983. Workarounds so far: a simulator device on the 26.5 runtime, or generating one MeshResource per part. Has anyone seen DRValidateIndices trapping on the 27.0 simulator?
Replies
1
Boosts
0
Views
615
Activity
2h
Logitech Muse never appears under Settings > General > Spatial Accessory — spatial tracking permission never granted
I'm building a visionOS 26 app that uses SpatialTrackingSession with .accessory tracking and AnchoringComponent.AccessoryAnchoringSource to track a Logitech Muse, following the pattern from the WWDC25 "Explore spatial accessory input on visionOS" session (and matching two independently published implementations I found). What works: The Muse pairs fine over Bluetooth, connects as a GCStylus (productCategory == .spatialStylus), GCStylusDidConnect fires, and button input (pressedDidChangeHandler) works correctly. What doesn't: SpatialTrackingSession.run() always reports .accessory as unavailable, so AnchoringComponent.AccessoryAnchoringSource(device:) never succeeds. Checking Settings > General > Spatial Accessory on the device shows the Muse isn't listed there at all — not even a battery/status entry. Settings > Privacy & Security > Spatial Accessory also never lists our app. What I've tried: Forgetting the Muse in Bluetooth and re-pairing Full power-cycle of both Bluetooth and the Muse itself Confirmed NSAccessoryTrackingUsageDescription is set and the Accessory Tracking + Game Controllers (Spatial Gamepad) capabilities are enabled in Xcode Confirmed via Apple's own docs, the WWDC session, and two developer write-ups that no special commissioning step beyond standard Bluetooth pairing should be needed For comparison, XR Putt (which also uses the Muse) works fine and does show up with a green checkmark under Spatial Accessory permissions on the same headset. Has anyone hit this specific "device never gets commissioned as a spatial accessory" issue, or knows what actually triggers that commissioning beyond generic Bluetooth pairing? Happy to share more logs/details.
Replies
1
Boosts
0
Views
702
Activity
1d
ARKit World Tracking Drift Regression on LiDAR-Equipped Devices - iOS 26.4+
Summary We have identified a reproducible world tracking drift regression in ARKit on LiDAR-equipped iOS devices running iOS 26.4 and later. A static virtual node anchored at the world origin visually drifts from its initial position as the user moves around a real-world scene, despite the scene remaining physically static. The same code produces stable, drift-free results on non-LiDAR devices running identical OS versions. Device & OS Observations Testing was performed across four devices on iOS 26.4 using the same application build and ARWorldTrackingConfiguration settings. Non-LiDAR devices — iPhone 14 and iPhone 15 — produced stable, drift-free tracking in all test runs. No world origin displacement was observed regardless of how long or how far the user walked. LiDAR-equipped devices — iPhone 14 Pro and iPhone 16 Pro — exhibited consistent, reproducible drift. A static node placed at the world origin visually shifted from its initial position as the user moved through the scene. The same devices were stable on earlier iOS versions, confirming this is a regression introduced in iOS 26.4. Technical Observations Nature of drift: A SCNNode placed statically at the ARKit world origin (SCNVector3(0, 0, 0)) visually displaces from its original position as the user walks around a static real-world scene. The displacement is not random — it accumulates directionally as the user moves, consistent with a sensor fusion or coordinate anchoring error. Trigger condition: The drift occurs during normal walking motion around a fixed point of interest, such as circling a parked vehicle. It does not appear when the device is held still. LiDAR specificity: The drift is exclusive to devices with a LiDAR scanner. Identical hardware configurations — same iOS build, same ARWorldTrackingConfiguration settings — on non-LiDAR devices produce no drift whatsoever. This isolates the regression to the LiDAR sensor's contribution to ARKit's internal Visual-Inertial Odometry (VIO) fusion pipeline. No API-level workaround found: There is currently no public ARKit API to selectively disable the LiDAR scanner's contribution to VIO. All available configuration-level options have been evaluated without resolving the drift. ARWorldTrackingConfiguration Options Evaluated The following configuration changes were applied individually and in combination. None resolved the drift on LiDAR devices: isAutoFocusEnabled = false — No improvement videoHDRAllowed = false (disabled) — No improvement planeDetection = [] (disabled) — No improvement sceneReconstruction = .mesh — No improvement worldAlignment: .gravity vs .gravityAndHeading — No improvement Minimal Reproduction Case The drift can be reproduced with a minimal ARKit scene: Create an ARSCNView with ARWorldTrackingConfiguration using default settings. Add a single static SCNNode (e.g., a small sphere or axes geometry) at SCNVector3(0, 0, 0) when the session starts. Run the app on a LiDAR-equipped device (iPhone Pro, iPad Pro with LiDAR) on iOS 26.4 or later. Walk in a circle around the node's approximate real-world position. Expected: The node remains visually fixed at its world position throughout the walkthrough. Actual: The node drifts from its initial position, increasingly displaced from its world origin anchor as walking continues. We want to know if Apple has made any internal updates to ARKit, particularly after the OS 26.4 upgrade. Thanks!
Replies
7
Boosts
8
Views
3.8k
Activity
1d
Production LiDAR capture app affected by ARKit world-tracking drift on iOS 26.4+ (re: thread 827240) mitigation guidance and fix timeline?
Hello, and thank you for taking questions this week. We develop DeepWalk, a production App Store application that captures LiDAR depth, RGB, and ARKit camera poses while field technicians walk continuous outdoor routes (sidewalk and infrastructure assessment). Recorded ARCamera.transform values are used downstream for 3D reconstruction, so world-tracking accuracy directly determines our output quality. Problem: Beginning in early April 2026 and coinciding with the iOS 26.4 rollout across our device fleet, we observe directional, accumulating pose drift in recorded trajectories during normal walking motion. The behavior matches the regression reported in forum thread 827240 (world-tracking drift on LiDAR devices on iOS 26.4+, absent on non-LiDAR devices running identical code). Link to existing post here: https://developer.apple.com/forums/thread/827240 Our configuration is minimal: a default ARWorldTrackingConfiguration with frameSemantics = [.sceneDepth, .smoothedSceneDepth] and no other options set, running on a bare ARSession with a Metal renderer (no SceneKit/RealityKit). We measure drift directly from per-frame ARCamera.transform values, so the displacement appears in ARKit's pose output itself rather than in any rendering layer. The drift accumulates with walking distance and does not occur when the device is stationary. We've observed this across a large variety of devices (iPhone 15/16/17 Pro/Pro Max) on all iOS versions 26.4 or greater. We've also observed this on iPad Pros with updated firmware. Thread 827240 additionally reports that disabling autofocus, HDR, and plane detection and varying scene reconstruction and world alignment did not resolve it. Question Because depth capture is our core function, we cannot avoid engaging the LiDAR scanner, and we're not aware of a public API that adjusts its contribution to world tracking. We'd be grateful for guidance on three questions: Is this a known issue in iOS 26.4 and later? Did the LiDAR contribution to world tracking change in that release? For apps that require sceneDepth, is there any recommended session configuration, video format, or capture-side practice that reduces the drift in the interim? Is a fix planned, and if so, should we expect it in an iOS 26.x update or the iOS 27 cycle? This affects how we manage our deployed device fleet. We're happy to provide trajectory datasets, additional sysdiagnoses, or anything else useful. We have quantified drift-per-distance data across OS versions.
Replies
7
Boosts
1
Views
1.5k
Activity
1d
WebXR Augmented Reality Module
On visionOS 26.5, I noticed the Safari Extension 'WebXR Augmented Reality Module' option. I've enabled this option in Safari Settings, and restarted Safari, but when I try to launch a WebXR experience that supports immersive-ar as option (for example this one: https://www.viverse.com/dPASDhP I don't see the option to Play in AR (like I do with a Quest 3S headset) Does anyone have any examples of websites that can make use of this new WebXR Augmented Reality Module feature flag in Safari, on visionOS?
Replies
3
Boosts
4
Views
1.6k
Activity
3d
Main Camera Error
An incomprehensible error occurs whenever I try to access Main Camera features. I have already acquired the Main Camera entitlement and granted the required permissions, yet the error persists. To make matters worse, even Apple’s provided sample code triggers the exact same issue. The error message is far too uninformative to diagnose the root cause, and I urgently need a resolution as this is completely blocking progress. Multiline [C:1-2] Error received: Invalidated by remote connection. failed to observe with mask <BLSXPCBacklightProxyObserverMask: 0x108accf80; didUpdateToState: YES; eventsArray: YES> error:<XPC error received on message reply handler (3:BSServiceConnectionErrorDomain) "The operation couldn’t be completed. XPC error received on message reply handler"> non-launching port is incompatible with service identifier "com.apple.PointerUI.pointeruid.default-service" nw_socket_copy_info [C1:2] getsockopt TCP_INFO failed [102: Operation not supported on socket] nw_socket_copy_info getsockopt TCP_INFO failed [102: Operation not supported on socket] Unable to find a compatible slice for binary archive at /System/Library/Frameworks/CoreImage.framework/ci_uberwrapper_bin.metallib *** Assertion failure in void ar_session_request_authorization(ar_session_t _Nonnull, ar_authorization_type_t, __strong ar_authorization_results_handler_t _Nonnull)_block_invoke(), session.m:1491, reason: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. <NSXPCConnection: 0x10e68dcc0> connection to service with pid 86 named com.apple.arkit.service.authorization: Exception caught during invocation of reply block to message 'requestAuthorizations:withReply:'. Exception: This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist. ( 0 CoreFoundation 0x0000000198b756d4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205972 1 libobjc.A.dylib 0x000000018d3a71cc objc_exception_throw + 88 2 CoreFoundation 0x0000000198b755c0 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1205696 3 ARKitCore 0x00000002bbd640cc 53FDDDD4-8DD1-3E86-ACB0-8867D29AF781 + 479436 4 CoreFoundation 0x0000000198b7c5b4 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1234356 5 CoreFoundation 0x0000000198b79558 686CE4EB-CE52-3DD5-8B23-EF6A6BBFACEA + 1221976 6 Foundation 0x000000019de4220c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2109964 7 Foundation 0x000000019de41dd0 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2108880 8 Foundation 0x000000019de4651c 61E78A52-3B44-3DCE-AF6D-92C33FA0FACA + 2127132 9 libxpc.dylib 0x00000002c43d5a00 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 150016 10 libxpc.dylib 0x00000002c43c5678 CE0BB604-B358-3F22-B1DA-FFACEFCA20AD + 83576 11 libdispatch.dylib 0x00000002c41045fc 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 108028 12 libdispatch.dylib 0x00000002c41083ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 123884 13 libdispatch.dylib 0x00000002c40f37c0 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 38848 14 libdispatch.dylib 0x00000002c40f4450 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 42064 15 libdispatch.dylib 0x00000002c40fe3ec 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 82924 16 libdispatch.dylib 0x00000002c40fdd18 4CF750D7-AC3B-3175-9863-2F059D3D3715 + 81176 17 libsystem_pthread.dylib 0x00000002c4364e34 _pthread_wqthread + 292 18 libsystem_pthread.dylib 0x00000002c4362910 start_wqthread + 8 ) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This app failed to request an authorization. The NSWorldSensingUsageDescription / NSHandsTrackingUsageDescription / NSAccessoryTrackingUsageDescription / NSMainCameraUsageDescription keys may be missing from this app's Info.plist.' *** First throw call stack: (0x198b756c8 0x18d3a71cc 0x198b755c0 0x2bbd640cc 0x198b7c5b4 0x198b79558 0x19de4220c 0x19de41dd0 0x19de4651c 0x2c43d5a00 0x2c43c5678 0x2c41045fc 0x2c41083ec 0x2c40f37c0 0x2c40f4450 0x2c40fe3ec 0x2c40fdd18 0x2c4364e34 0x2c4362910) libc++abi: terminating due to uncaught exception of type NSException Xcode has killed the LLDB RPC server (pid: 2210) to allow the debugger to detach from your process. BlockQuote visionOS 27 beta 5
Replies
5
Boosts
0
Views
1k
Activity
4d
How to use ClothGrabComponent?
I've simulated a piece of cloth using https://developer.apple.com/documentation/realitykit/physics-cloth-simulation I would like to grab the cloth like what I can in Marvellous Designer. I found the ClothGrabComponent at https://developer.apple.com/documentation/realitykit/clothgrabcomponent However, after I added the ClothGrabComponent to the cloth entity, I am still not able to grab it. Is there any sample code to show me how to grab the cloth? Thank you.
Replies
1
Boosts
0
Views
648
Activity
5d
BloomComponent lags behind the camera on iOS/macOS but not visionOS
BloomComponent (RealityKit 27) lags behind the geometry that produces it while the camera moves. The halo trails the emissive geometry by roughly one to three frames (eyeballed) and snaps back once the camera stops. Video attached — it's obvious at normal playback speed. visionOS renders the identical scene correctly. Only iOS and macOS / Mac Catalyst lag, which points at the non-visionOS compositing path rather than the effect itself. Filed as FB23960052. This should be fixed before 27 ships — bloom is unusable for anything with a moving camera in its current state on those platforms. Ruled out: Scope. Identical with BloomComponent(scope: .hierarchical) and .unbounded. .unbounded computes no per-entity screen-space bounds, so stale bounds are not the cause. Input handling. The camera is RealityKit's own .realityViewCameraControls(.orbit) — no gesture code of mine involved. Per-frame component writes. BloomOptionsComponent is set once and untouched during the drag. Geometry and camera transform. Both track perfectly; only the glow lags. Reproducer is ~130 lines, no assets — five emissive spheres inside a large inward-facing dark sphere: var material = PhysicallyBasedMaterial() material.baseColor = .init(tint: .black) material.emissiveColor = .init(color: .cyan) material.emissiveIntensity = 4 // ... root.components.set(BloomComponent(scope: .unbounded)) var options = BloomOptionsComponent() options.strength = 1 options.threshold = 1 options.blurRadius = 1 root.components.set(options) shown in: RealityView { content in content.camera = .virtual content.add(root) } .realityViewCameraControls(.orbit) Xcode 27.0 beta 4, iOS 27.0 SDK, Apple Silicon. Affected: iOS 27, macOS 27 / Mac Catalyst 27. Not affected: visionOS 27 (immersive space). Full project and screen recording attached to my radar.
Replies
4
Boosts
0
Views
1.8k
Activity
6d
GaussianSplatComponent — stale visibility set after close-range camera approach
Title: GaussianSplatComponent — stale visibility set after close-range camera approach (iOS 27 beta 4, still unfixed) Filed this as a radar during beta 3; still reproduces unchanged on beta 4, so posting here in case anyone from the RealityKit team can confirm it's tracked before this ships. Radar: FB23749559 (filed against beta 3, re-verified on beta 4) Summary On iOS, GaussianSplatComponent keeps a stale visibility set after the camera moves close to and then away from the splat. Moving the camera in close causes a large portion of splats to be culled — expected. But pulling the camera back out does not repopulate them. The visible set stays stale for as long as the current camera transform is held. The missing splats only return once the camera transform changes again — any pan/orbit/zoom delta triggers a recompute and the full splat snaps back. Net effect: after any close inspection, the splat renders permanently partially-culled until the user happens to nudge the camera. The recovery-on-movement / no-recovery-when-stationary signature suggests the iOS render path recomputes splat residency on camera-transform change events rather than every frame. This also reproduces in AR mode, with a completely different camera driver. Since the only thing the two paths share is the splat residency code, that's where the stale set has to be coming from — which also means there's no app-side fix available to me. The camera transform in AR is driven by ARKit, not by my code, so I can't force the perturbation that would otherwise work around it. visionOS 27, running the same component, is unaffected — the visible set is re-evaluated per frame and the splat stays complete. Steps to reproduce Attach a GaussianSplatComponent (~400k splats) to an entity in a bounded RealityView with orbit controls. Move the camera close until a visible portion of the splats is culled. Move the camera back out so the full splat is within the frustum. Hold the camera still. Expected: full splat visible once pulled back. Actual: previously-culled splats stay missing while the camera is stationary; they reappear only on the next camera transform delta. Configuration iOS 27 beta 4 (originally filed against beta 3) iPhone 16 Pro ~400k splats Bounded RealityView + orbit controls; also reproduces in AR Screenshots attached: full splat, stale-culled state after pull-back, and the close-range view that triggers it. This is still present in beta 4 and in the public beta. I'd like to know whether it's tracked, or whether I should plan around splats being unreliable on iOS 27. Thanks!
Replies
5
Boosts
0
Views
1.3k
Activity
6d
visionOS 27: allAnchors and anchorUpdates disagree on WorldAnchor transforms (different coordinate frame)
After updating to visionOS 27.0, WorldTrackingProvider.allAnchors returns WorldAnchor.originFromAnchorTransform values that do not match what anchorUpdates delivers for the same anchor, read at the same moment, both isTracked == true, with no .updated event in between. The difference is a rigid frame change, not drift: Same vertical offset on every anchor: -1.173 m. Identical across two apps and two days with the head at 1.13 m and 1.22 m, so a constant, not head height. Horizontal deltas vary per anchor, but inter-anchor distances are preserved, so it is a yaw plus translation. Stable over time, same across launches. anchorUpdates matches the ImmersiveSpace frame RealityKit draws in. allAnchors does not. On visionOS 26 both agreed. Same anchor, same instant: allAnchors (0.1486, -0.7143, 0.2231) anchorUpdates (0.1316, 0.4590, -1.0955) device pose (0.0073, 1.2195, -0.0038) Four other anchors, delta = allAnchors - anchorUpdates: ( 2.6933, -1.1733, -0.2514) (-0.0810, -1.1733, 1.1339) ( 0.1878, -1.1733, 0.9891) ( 2.6289, -1.1733, -0.1809) In our app this showed up as a placed model vanishing after a scene switch: we re-pinned the model root from allAnchors, and it landed below the floor behind the user. A standalone sample (one provider, one anchor, a green sphere from anchorUpdates, a red sphere from allAnchors) shows two spheres on visionOS 27.0, the red one below the floor: added F21942C4-… at (-0.031, 0.712, -0.976) update added F21942C4-… tracked=true at (-0.031, 0.712, -0.976) MISMATCH delta (-0.211, -1.173, 1.444) m allAnchors (-0.243, -0.461, 0.468) anchorUpdates (-0.031, 0.712, -0.976) tracked=true device (0.000, 1.129, -0.001) Questions: Is allAnchors now intentionally expressed in a different frame on visionOS 27, for example related to the new ARKitCoordinateSpaceProviding / coordinateSpace(correction:) API? The documentation for allAnchors does not mention it. If so, which frame is it, and how should it be converted to the ImmersiveSpace frame? Workaround we use: read transforms only from anchorUpdates into a per-id cache and never position from allAnchors. Filed as FB24799304. Minimal repro project attached to the feedback.
Replies
1
Boosts
0
Views
115
Activity
6d
Camera specs of world-facing tracking cameras.
We are working on a very space constrained custom spatial accessory. Since the form factor of the device is small, we need to add small LEDs producing small blob diameters with accordingly small distance between them. To better estimate the amount, size, and distance of the LEDs it would help to know which image sensors and field of view the six world‑facing tracking cameras have.
Replies
1
Boosts
0
Views
786
Activity
1w
Update entity state across devices through SharePlay on VisionOS
I am trying to create an app where I have multiple users modifying an entity at runtime similar to collaborative digital sculpting using SharePlay on VisionOS. When creating a collaborative experience using other 3D engines I have utilized an external server that devices send and update their state from serving as a global "source of truth". This doesn't seem to be the intended workflow for SharePlay which seemingly only allows for device to device messaging. While this works for a variety of apps it doesn't provide a smooth experience when multiple users are interacting with and manipulating the same 3D entity. Is there an intended way to do this for SharePlay? Would the best practice be to have one user serve as a host and establish a "source of truth" based on their app state? The closest example in documentation I have found is an example for using SharePlay in a drawing app. When a user is drawing they send the information for their realtime stroke through unreliable SharePlay messages and then, when they've finished, they send a reliable message with the final line. This technique makes sense for the given example since there is no scenario where multiple users would be drawing the same line, but it seemingly falls short for my use case since multiple users can sculpt in the same area of a given mesh. What would the intended approach be to achieve this with SharePlay? Is it simply a limitation of this type of networking? Thanks!
Replies
1
Boosts
0
Views
1.2k
Activity
1w
RealityKit cloth: reducing mesh collider air gap without penetration
I’m testing RealityKit cloth for garment-like cloth over an authored human-shaped triangle-mesh collider. The issue also occurs with a static collider, so it does not appear to depend on character animation.. Contact is stable and runs interactively, but I cannot achieve close surface contact without sacrificing robustness. The cloth mid-surface remains visibly several millimetres from the authored collider, especially around curved or narrow areas such as the head, arms, and hands. Transparent rendering of visualMesh confirms this is not just render thickness. Reducing ClothMeshShape.bias reduces the gap, but eventually causes interpenetration around difficult geometry. Increasing it prevents penetration but makes the cloth visibly float. I’ve also checked scale, winding, normals, reasonably even triangles, timestep, and solver iterations. I want to avoid inflating the collider or adding anatomical proxy shapes. Is bias the only physical-clearance control, or does RealityKit apply an additional contact radius derived from simulation-mesh resolution? Is local conforming refinement the recommended way to achieve roughly 1–2 mm clearance while retaining robust real-time contact? Are there any specific topology requirements beyond evenly sized triangles? I can provide a minimal reproduction and screenshots if helpful although not sure I can share screenshots here?
Replies
3
Boosts
0
Views
3.2k
Activity
1w
Questions about RoomPlan, Room API, USDZ/STEP comparison, and extended spatial scanning
Dear Apple Developer Team, I would like to ask a few questions related to RoomPlan, Room API, USDZ export, and possible future spatial scanning workflows: Can the RoomPlan or Room API load an existing USDZ room model and compare it in real time with a new live scan of the same space? Is Apple considering extending RoomPlan beyond indoor rooms, for example to scan outdoor areas, house exteriors, terrain, and simple building volumes with rough dimensions? Will RoomPlan support multi-floor continuous scanning, custom object detection elements, and reliable export of the scanned result to USDZ for further CAD/BIM workflows? Is it possible to create a real-time comparator between a reference object in USDZ or STEP format and a physical object being scanned live, so that deviations in geometry or dimensions can be detected during scanning? Best regards, Ivo Saina
Replies
4
Boosts
0
Views
866
Activity
1w
Is there any technical documentation available for the iPhone’s LiDAR scanner?
Hi, I’m looking to develop an app that uses the iPhone’s LiDAR scanner and am trying to decide which model to buy. Is there any relevant technical documentation available on its capabilities, such as accuracy, resolution and range? I’d also like to understand whether there have been any improvements between generations, particularly with the iPhone 18 Pro. Any documentation or pointers in the right direction would be appreciated. Thanks!
Replies
0
Boosts
0
Views
279
Activity
1w
Best practice for connecting Mac-side USDZ preparation with exact-asset Vision Pro verification?
I’m developing two complementary developer tools: Reality Prep Pro on macOS and Reality Prep Preview on visionOS. The workflow I’m trying to solve is: Mac: prepare and validate a USDZ Vision Pro: verify that exact USDZ on-device Mac: bring the device evidence back and associate it with the exact asset revision I currently use the USDZ’s SHA-256 as the authoritative identity. If the asset bytes later change, the previous Vision Pro evidence is treated as stale and a re-test is recommended. Existing USDZ→USDZ preparation is package-preserving, so non-texture payloads such as animation/rigging data are not unnecessarily re-authored. I’d be interested in hearing from other RealityKit / Reality Composer Pro developers: Is this similar to how you currently handle asset provenance between desktop preparation and real-device testing? Are there any Apple frameworks or established visionOS practices you would recommend for strengthening this kind of revision-to-device-evidence workflow?
Replies
0
Boosts
0
Views
501
Activity
1w
Generic spatial accessory receives LED reports but never produces an AccessoryAnchor
We are developing a generic spatial accessory for Apple Vision Pro using the “Working with generic spatial accessories” sample and a trained .referenceaccessory file. Environment: visionOS: [version/build] Xcode: [version/build] Hardware: Apple Vision Pro + nRF52840-based BLE accessory IMU: 6-axis IMU, 400 Hz HID reports LED constellation: 8 x 850 nm IR LEDs, driven through MOSFETs The BLE/HID and timing path appears to be working: The accessory connects successfully over BLE. ASA/HID enumeration succeeds. The firmware receives continuous LED Output Reports from visionOS. The Accessory Tracking Clock reaches READY. PWM scheduling reports no late events and no queue overflow. In the Accessory Tracking Debug View, multiple independent IR LED blobs are visible and stable at approximately 0.7 m. At least 4 LEDs are visible from normal viewing angles. The app resolves the .referenceaccessory and starts AccessoryTrackingProvider. However, ARKit never produces an AccessoryAnchor. We also have no diagnostic signal indicating whether the failure occurs at: LED candidate-point detection, constellation matching against the .referenceaccessory, or IMU/optical fusion. Questions: Is there a supported diagnostic API, log category, or Debug View indicator that distinguishes candidate-point rejection from constellation-match or fusion rejection? Are there known acquisition prerequisites in the current visionOS beta beyond the published Accessory Design Guidelines? Can Apple recommend a minimal validation procedure using the generic spatial accessory sample to confirm that the LED constellation is accepted by the tracking system? Is this expected to work in a volumetric window, or are there additional lifecycle/session requirements that can prevent initial acquisition? I can provide sanitized HID LED-report logs, timing data, Debug View captures, and a minimal reproducible project if useful.
Replies
10
Boosts
0
Views
2.8k
Activity
1w
Physical space lighting (SurroundingsLight) never lands on sloped ceiling surfaces that are present in the scene reconstruction mesh
Summary A SpotLightComponent with SpotLightComponent.SurroundingsLight() (and a ProjectiveTexture) lights the flat part of my ceiling and the walls, but never the two sloped ceiling surfaces of a converted attic room, at any angle, intensity, attenuation radius or falloff. The lit area stops in a hard straight line exactly where the flat ceiling meets each slope, even though the beam clearly extends past that line. The sloped surfaces are fully present in SceneReconstructionProvider's mesh, and their faces point into the room. So this doesn't look like a scene understanding coverage problem. It looks as if physical space lighting only considers surfaces classified as horizontal ceiling, wall or floor, and skips slanted ones. Steps to reproduce Open a mixed immersive space in a room with a sloped ceiling (a converted attic, or any vaulted ceiling). Add an entity with SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10), SpotLightComponent.SurroundingsLight(), and a SpotLightComponent.ProjectiveTexture carrying a labelled grid texture so the beam's extent is visible. Place it near the floor under the wearer and aim it straight up. Run SceneReconstructionProvider alongside and draw each MeshAnchor as a wireframe (or as a solid material with faceCulling = .back) to confirm the mesh covers the slopes and faces inward. Expected The projected pattern conforms to every surface in the scene understanding mesh inside the cone, including the sloped ceiling, as it does for the flat ceiling and walls. Actual The pattern appears only on the flat ceiling strip and on the walls. Both slopes stay completely dark, with a hard straight cut at the crease. Tilting the light to aim directly at a slope, raising intensity to 200,000, and changing attenuation radius and falloff make no difference. The wireframe shows continuous mesh across the crease and onto the slopes; the back-face-culled solid view shows the slope faces pointing into the room. Screenshots attached: (1) the projected grid stopping at the crease, (2) the reconstruction wireframe covering the slopes, (3) the solid front-faces-only mesh showing the slopes are front-facing. (4) an additional image of the projected grid with both ceiling and walls in view Code let spot = Entity() spot.components.set(SpotLightComponent.SurroundingsLight()) var light = SpotLightComponent(color: .white, intensity: 40_000, innerAngleInDegrees: 44.5, outerAngleInDegrees: 45, attenuationRadius: 10) light.attenuationFalloffExponent = 0.5 spot.components.set(light) spot.components.set(SpotLightComponent.ProjectiveTexture(texture: gridTexture)) spot.look(at: from + SIMD3(0, 1, 0), from: from, upVector: SIMD3(0, 0, -1), relativeTo: nil) root.addChild(spot) Questions Is physical space lighting limited to horizontally and vertically classified surfaces by design? If so, could this be documented on SurroundingsLight, and is there a way to include slanted surfaces? If not by design, is there anything an app can do to have the light use the full reconstruction mesh?
Replies
2
Boosts
0
Views
1.1k
Activity
2w
Is there currently a way to import 3D Gaussian Splatting data into Reality Composer Pro?
Hello, As far as I understand, RealityKit now supports 3D Gaussian Splatting through GaussianSplatComponent. I would like to ask whether there is currently a way to use 3D Gaussian Splatting data directly in Reality Composer Pro, for example by importing a .ply file containing Gaussian splats. More specifically: Can Reality Composer Pro import a 3D Gaussian Splatting .ply file? Is there a way to create or assign a GaussianSplatComponent to an entity directly within Reality Composer Pro? If this is not currently supported, is there any recommended workflow for creating a .usdz containing Gaussian Splatting data that can be opened in the Preview app on iPhone/iPad and displayed as an AR object? I tried opening the sample plant.usdz on an iPhone using the Preview app. However, the AR preview appears to be empty, so it seems that the Preview app may not currently support rendering the Gaussian Splatting content contained in that USDZ. My desired workflow would be: Gaussian Splatting .ply → Reality Composer Pro / USDZ → iPhone Preview → AR Is this workflow currently possible? If not, is there an official or recommended way to package a GaussianSplatComponent and its associated Gaussian splat data into a USD/USDZ file for AR preview on iOS? If the above functionality is not currently supported, does the development team have any plans to add or improve support for these features in future versions of Reality Composer Pro or the Preview app? Any clarification regarding the current capabilities, limitations, or future plans would be greatly appreciated. Thank you!
Replies
3
Boosts
0
Views
2.4k
Activity
2w
Anchoring Invalid
Hello. I trained an ObjectTracker.referenceobject using ML, and am using it in RCP. After adding Anchoring to the entity, I set the Target mode to Object and applied the trained model data ObjectTracker.referenceobject. However, there is no tracking effect after building for VisionPro. Are any permissions required? What should I do?
Replies
3
Boosts
0
Views
2.5k
Activity
2w