Working with augmented reality (AR) is not all that different from implementing any other piece of Native functionality. Using just a few widgets, you can add two-or-three dimensional AR to your native mobile application. In this how-to you will make a 3D cube by dragging and dropping widgets into your Mendix app and then configuring them.
Before starting this how-to, make sure you have completed the following prerequisites:
- Have an Android mobile device that can support ARCore
- Install Google Play Services for AR on your device
- Install the Make It Native app on your Android mobile device for testing purposes
3 Embedding Widgets in Your App
Create a new app by following these steps:
- Open Mendix Studio Pro. Select File > New Project , and then select the Blank App.
- Click Use this starting point.
- Name your app Hello World and click Create app to close the dialog box.
- Right-click MyFirstModule and click Add page.
- Click Native mobile at the top of the dialog box.
- When prompted with the No NativePhone profile exists. Do you want to create it? dialog box, click Yes.
- Select the Blank > Blank page and click OK.
- On this new page, remove the empty container so you can start with a completely blank page.
You will begin by embedding the following widgets into your Mendix app:
Embed the widgets in this order:
- Drag and drop ARContainer onto a native mobile page.
- Drag and drop ARImage tracker into ARContainer.
Drag and drop ARCube into ARImageTracker:
Very little configuration is necessary to make your AR function. All you need is an image tracker. A tracker is an image your AR can track and put AR effects on top of. Most images can serve as a tracker, provided they have enough contrast. Complex images make for the best tracker, because they are easier for your device’s camera to track. Start by adding an image to ARImageTracker:
- Double-click your ARImageTracker widget.
- Click Image > Edit.
Select a tracker from an image collection. This image will be the foundation of your AR application and is what your camera will track:
Click Run in Mendix Studio Pro to automatically refresh your Make It Native app. Now your Hello World AR app is up and running. You should now see a simple white cube rendered on top of a tracker:
Now that you have a simple scene running, you will a closer look at the widgets you used to understand what they do.
This widget starts an AR scene: the video feed you see on AR apps. It is named ARContainer because this widget will hold all your other AR widgets. It is the foundation of every Mendix AR mobile application:
Selecting Enable autofocus > No can give a slight benefit to performance, but in most cases you should keep Yes selected.
The ARImageTracker widget allows you to track an image. Like ARContainer, this widget can contain other AR widgets. All widgets you put into ARImageTracker will stick to the image. That means that every object (ARCube, ARSphere, and others) will follow or track the image.
Image — in the General tab, you can provide your own image to this widget by clicking Image > Edit. While it is possible to add any image, some images work better for AR tracking than others. Images can best be tracked when they have high contrast and several edges:
It is possible to use a more conventional image, like a company logo, as long as it has enough contrast and edges. If your image does not posses these qualities, you will notice AR elements floating in incorrect places and motion tracking failures as you move your camera. It is also possible that your image will not be recognized at all.
Orientation — this dictates the orientation of 3D objects on your tracker, not the orientation of the tracker itself. Leaving it on Up will make your 3D object appear right side up when you have it lying on a table.
Your standard cube will not look different when turned around. To show orientation, this tutorial has temporarily added the following texture was added (see the Material section below to learn how to add your own texture to an object):
Here is a cube with the Up orientation:
Here is a cube with the Left orientation:
4 Configuring Your ARCube Widget
The ARCube widget will places a cube into the scene, specifically onto the ARImageTracker widget. There are several properties you can configure in this widget to customize its behavior.
Having the ARCube widget in the ARImageTracker widget will render a white cube the same size as the image tracker directly on top of it. Next you will configure ARCube’s properties.
The General tab contains basic properties for configuring position:
In the General tab you can configure Position, Rotation, and Scale (X stands for left/right, Y for up/down, and Z for depth):
Position — the position relative to the image tracker. For example, since the position is relative to the tracker size, set X to 1 to make your object appear right beside the tracker instead of on top of it:
Here is a cube in Position (1,0,0):
Here is a cube in Position (0,1,0):
Here is a cube in Position (0,0,1):
Rotation — the rotation of the cube in 360 degrees. Set Rotation X to 45 to see your cube rotated 45 degrees on its X axis:
Here is a cube with Rotation (0,0,0):
Here is a cube with Rotation (45,0,0):
Scale — the size of the object, also relative to the tracker it is embedded in. When the Scale is set to (0,0,0) the object will be invisible. Because 1 is exactly the size of the tracker, set Scale X, Y and Z to 0.5 to see your cube shrink in the tracker:
Here is a cube with Scale (1,1,1):
Here is a cube with Scale (0.5,0.5,0.5):
The Material tab contains properties for configuring appearance:
Texture — an image you can place on the cube by clicking Edit. The image will appear on each face of the cube. Put the example tracker here as a texture to end up with this colorful cube (note that the cube now ignores any Color value — if you want to make your cube one solid color, make sure the Texture is set to none):
Color — the color of the cube. You can have either a Texture or Color, but not both. White is the standard color. Change the value to green to make your cube appear this way:
Opacity — the clarity or opacity of the cube. 1 is fully opaque, while 0 is fully transparent. Change the color of your cube back to white and change the opacity value to 0.5 to see your cube become partly transparent:
Lighting type — the way light from the scene will fall on your object.
- Phong, Blinn, and Lambert are standard configurations of adding light and shadows to your objects. They are also have relatively slight processor power requirements.
- Constant means no light is added — just pure colors are shown.
- PBR (Physically Based Rendering) is the most advanced. PBR takes the entire scene into account when creating its lighting, the intricacies of which are beyond this how-to. Change between the various Lighting type options to see them in action:
Here is a cube with Lighting type > Phong:
Here is a cube with Lighting type > Constant:
Interaction is where Mendix AR gets more advanced. These properties dictate every way you can interact with your object. Enabling any of these will also allow you to catch the events they generate in the Events tab:
Physics — dictates how your object interacts with other 3D objects. Enabling it will not show an immediate visible difference, but it will make it possible for other objects to collide with this object.
- Physics type — how the object should physically behave. Selecting Kinematic will make your object move only when manipulated. Dynamic objects will react to everything. Static objects will never move.
- Gravity — sets if the object should be affected by gravity. Setting it to ‘Yes’ will make your object fall until it meets another object with physics enabled. To put this information into practice, select Enable Physics > Yes, select Physics Type > Dynamic, and select Use Gravity > Yes to enable your cube’s physics:
Dragging — allows a user to manipulate an object by dragging it with their finger. This property gives users a very instinctive way to move objects
Pinching — can be used to scale the object, similar to zooming in and out on a maps widget. Select Enable Pinching > Yes, select Enable Pinch to Scale > Yes, and set the Scale Factor to 1, then pinch out on your object to examine it in detail.
The Events tab appears this way:
Events on 3D objects work like any other event on a widget. The main difference is that On Drag, On Pinch, and On Collision events will only be called when they are enabled in the interaction tab.
The Common tab appears this way:
The Common tab has two important properties:
- Name — is used internally in all AR Widgets and must be unique. It can be changed, but we recommend keeping its generated name.
Visible — is used to dictate the visibility of the AR component. This can be conditional just like most other widgets:
The Appearance tab appears this way:
Currently the properties in the Appearance tab have no influence on AR widgets. This tab can be ignored.