Back to Blog

Learn to Code in Augmented Reality

Learn to Code in Augmented Reality

Using ARKit to put Rapid Router in the real world

Code for Life’s first open source game, Rapid Router, introduces primary school children to the world of coding as they program a van using Blockly and Python to navigate different road maps, visiting houses along the way to deliver groceries (if you haven’t already, give it a try and tell us what you think!).

In June, we passed the 100,000 registered users mark, and our game is now helping more children around the world get to grips with coding concepts. However, there are always ways in which we can improve, so our group of committed volunteers are constantly brainstorming ideas to experiment with. To make sure Rapid Router keeps up with the ever changing needs of today’s teachers, we also regularly check in with them to gather all the feedback we can. Over time we have found that one comment kept on coming up in our discussions:

“You often have to ask the child to work out the orientation by imagining they’re in the driver’s seat. For some scenarios like the one below, students find that difficult.”

Upside down van route

Now that the van is upside down, which way is left and which is right?

We thought this could be an interesting problem to experiment with, but how could we implement it in practice? Should we navigate using compass points i.e. North, South, East and West? That could cause other related problems and we wanted to keep everything from the point of view of the van, as it seemed more intuitive.

Ultimately, we wanted to make sure that the student could focus on coding, rather than getting distracted by the problem domain we teach them in.

Introducing ARKit!

Earlier this year Apple announced a framework that allowed developers to render 3D scenes in real time, in real world environments. In addition to that, it can detect surfaces, light sources and dimensions, just by using the hardware available on an iOS device. Pretty neat if you ask me!

ARKit allows developers and enthusiasts, who don’t necessarily have a thorough understanding of the low-level concepts behind augmented reality, to try their hand at some really creative projects. Here are two of my favourite demos so far:

It’s inspiring to see what the community has made in a short amount of time. Hyped up, we got started with trying it ourselves.

We thought that ARKit would be the perfect candidate for an experiment to see if we could help children to visualize the van’s position. This is partly because ARKit is so easy to get started with, which is important given that fact that Code for Life is a volunteer based initiative. We could potentially put a Rapid Router level on the floor or a table, and let children move around the roads themselves. Not only does this mean they can re-orientate themselves naturally, according to which way the van is facing, but hopefully they will also find it pretty cool!

How we did it

If you have read the previous post I wrote with Jacq Train about building the Rapid Router app for the iPad, you will know that we are currently using Unity. This meant converting our app to take advantage of ARKit was pretty simple. All we had to do was:

Install the ARKit plugin

Unity and Apple have provided an easy to install plugin for ARKit that you can download from the asset store. It contains the ARKit framework, some sample scenes and a scene that you can build on iOS that allows you to run your augmented reality project from within Unity. This is a lot faster than exporting your scenes to iOS and running it from there each time you want to try your code out. I recommend following the included tutorial to help you out.

Listen to the ARPlaneAnchorEvents

In order for the demo to be realistic, we needed to place the level’s map onto something flat in the real world. To do this I hooked into:

ARAnchorAddedEvent , ARAnchorUpdatedEvent , ARAnchorRemovedEvent 

I also kept track of the planes detected, just in case the one I was using was removed.

UnityARSessionNativeInterface.ARAnchorAddedEvent += AddAnchor;
 UnityARSessionNativeInterface.ARAnchorUpdatedEvent += UpdateAnchor;
 UnityARSessionNativeInterface.ARAnchorRemovedEvent += RemoveAnchor;

public void AddAnchor(ARPlaneAnchor arPlaneAnchor) {
 anchors.Add(arPlaneAnchor.identifier, arPlaneAnchor);
 if (currentAnchor != "") {
 return;
 }
 UpdateBoardWithAnchor(arPlaneAnchor);
 currentAnchor = arPlaneAnchor.identifier;
 }

public void UpdateAnchor(ARPlaneAnchor arPlaneAnchor) {
 if (anchors.ContainsKey (arPlaneAnchor.identifier)) {
 ARPlaneAnchor arpag = anchors [arPlaneAnchor.identifier];
 anchors [arPlaneAnchor.identifier] = arPlaneAnchor;
 if (currentAnchor == arPlaneAnchor.identifier) {
 UpdateBoardWithAnchor(arPlaneAnchor);
 }
 }
 }

public void RemoveAnchor(ARPlaneAnchor arPlaneAnchor) {
 if (anchors.ContainsKey (arPlaneAnchor.identifier)) {
 ARPlaneAnchor arpag = anchors [arPlaneAnchor.identifier];
 anchors.Remove (arPlaneAnchor.identifier);
 if (currentAnchor == arPlaneAnchor.identifier) {
 // Find another plane to put the map on
 }
 }
 }

Move the Board to the Plane Anchor

The only thing left to do was to put the map in the same position and scale as the plane. I positioned it in the centre of the plane and scaled it using the ARPlaneAnchor’s extent.

public void UpdateBoardWithAnchor(ARPlaneAnchor arPlaneAnchor) {
	GameObject board = GameObject.Find("Board");
	board.transform.position = UnityARMatrixOps.GetRotation(arPlaneAnchor.transform);
	board.transform.rotation = UnityARMatrixOps.GetRotation(arPlaneAnchor.transform); 

	Vector3 center = GetCenter(arPlaneAnchor);
	
	var scalingFactor = 0.11f;
	var scale = Mathf.Min(arPlaneAnchor.extent.x, arPlaneAnchor.extent.z);
	board.transform.localScale = new Vector3(scale * scalingFactor, scale * scalingFactor, scale * scalingFactor);

	//convert our center position to unity coords
	board.transform.localPosition = new Vector3(-center.x, center.y, -center.z);
}

Revel in ARKit glory

That’s pretty much it! Below is the completed demo:

What’s next?

We felt that ARKit could have great potential for a future Rapid Router iOS app. The layout feels intuitive, and we think that children will love the fact that they can play the game in a real world setting. The next step is user testing, getting an opinion from a child’s point of view, to see if they agree with us.

It’s always important to consider emerging technologies. Testing out ARKit has been simple, satisfying and, most importantly, fun!

Niket Shah, Mobile Developer and Code for Life volunteer

Code for Life is an open-source initiative by Ocado Technology. We exist to help shape technology’s future.

Scroll Up