{"_id":"5b6893dccbee3a0003551116","project":"5a065a6134873d0010b396ab","version":{"_id":"5b689070f3c8dc0003c6d1df","project":"5a065a6134873d0010b396ab","__v":0,"forked_from":"5b4e90b9d9ea0b00031c6194","createdAt":"2018-04-18T18:19:34.288Z","releaseDate":"2018-04-18T18:19:34.288Z","categories":["5b689070f3c8dc0003c6d1a4","5b689070f3c8dc0003c6d1a5","5b689070f3c8dc0003c6d1a6","5b05923ea5a2f9000357b452","5b05f793c2c86c0003cbe414","5b689070f3c8dc0003c6d1a7","5b689070f3c8dc0003c6d1a8","5b689070f3c8dc0003c6d1a9"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.9.0","version":"1.9.0"},"category":{"_id":"5b689070f3c8dc0003c6d1a9","project":"5a065a6134873d0010b396ab","version":"5b689070f3c8dc0003c6d1df","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2018-05-24T14:46:44.164Z","from_sync":false,"order":3,"slug":"enhanced-scenes","title":"Enhanced Scenes"},"user":"57bb7defafc18c0e00529cf1","githubsync":"","__v":0,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2018-08-06T18:30:52.960Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":999,"body":"ViroCore also enables developers to enhance their applications with interactive 2D Android views by rendering them in AR or VR! As such, developers can easily build their familiar XML layouts, Android Buttons and ListViews in Android, and ViroCore will do all the heavy-lifting of rendering those views in a 3D space.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/a2bbdd3-ezgif-5-4934b4cfe2.gif\",\n        \"ezgif-5-4934b4cfe2.gif\",\n        480,\n        270,\n        \"#9bb050\"\n      ]\n    }\n  ]\n}\n[/block]\nTo use 2D Android UI components in VR: \n1. Create an AndroidViewTexture with a width and height size representing your Android layout.\n2. Attach your Android layout view to the AndroidViewTexture.\n3. Apply this texture to any geometry you want to render the Android views on - a Cube, a Sphere or a Quad! \n\nYou can also refer to the example code below for a more detailed implementation walkthrough.\n### View Sizes and Hardware Support\nWhen constructing an AndroidViewTexture in Virocore, it is important to consider view sizing limitations - that is, the area of your Android View layout that is rendered before it gets clipped. \n\nBy default, ViroCore's AndroidViewTexture will render AndroidViews onto a Viro Texture without any hardware acceleration. This ensures maximum compatibility with most Android OS versions, while guaranteeing correct view interaction behavior. However, there is a **memory size limitation** with this approach, resulting in potentially smaller viable width and height sizes for the texture. \n\nTo check if your desired view size works with software acceleration, you can also use the helper function **AndroidViewTexture.suuportsSoftwareSurfaceOfSize(Context, desiredWidth, desiredHeight)** \n\nElse, developers can also force the AndroidViewTexture to use hardware acceleration, enabling **larger texture sizes**. Although most basic Android controls function properly with this mode, it is important to note that some views may have unpredictable behavior, particularly when handling user interaction.\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"body\": \"Note: If you see a black texture instead of your expected Android views, this is most likely an indication that the set width and height pixel values on your AndroidViewTexture has exceeded the device's capability to render them.\"\n}\n[/block]\n### Events and Interactivity\nUsers can also interact with and click on Android views that are attached to AndroidViewTextures in 3D (for example clicking on an Android Button with a VR reticle). \n\nTo achieve this behavior, developers can use the **dispatchClickStateEvents API **on an AndroidViewTexture to map a Viro click event to a pixel coordinate within the bounds of the AndroidViewTexture's size. With that API, ViroCore then effectively plumbs through those click events to be triggered normally as if it were and Android click event being passed onto your Android views. Note that the mapping of any click event's position to an AndroidViewTexture's rectangular layout will be determined by the geometry shape that you have applied that texture to (however we have made it easy to do so for flat rectangular surfaces, as shown below).\n\nFor convenience, developers can also use the AndroidViewtexture.getClickListenerWidhQuad(quad) helper function. This helper function helps construct a Viro ClickListener **that maps Viro clicks performed on the given quad to Android touch events**, which will then be automatically applied to the underlying view. This also takes into account the transformations that are applied to the quad, and as well as the size of the underlying texture.\n\n### Example Code:\nIn this example, we'll be attempting to render our Android layout shown below in AR. This layout will have an Android image, Android Text and an Android Button, all encapsulated within a RelativeLayout.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/4d4c38c-button.png\",\n        \"button.png\",\n        1080,\n        2220,\n        \"#090909\"\n      ],\n      \"caption\": \"activity_main.xml\"\n    }\n  ]\n}\n[/block]\nThe representation of the above view in xml:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"<?xml version=\\\"1.0\\\" encoding=\\\"utf-8\\\"?>\\n<RelativeLayout\\n    xmlns:android=\\\"http://schemas.android.com/apk/res/android\\\"\\n    android:id=\\\":::at:::+id/activity_main\\\"\\n    android:layout_width=\\\"match_parent\\\"\\n    android:layout_height=\\\"match_parent\\\"\\n    android:background=\\\"#99FFFFFF\\\">\\n\\n    <TextView\\n        android:id=\\\"@+id/custom_title\\\"\\n        android:layout_width=\\\"wrap_content\\\"\\n        android:layout_height=\\\"wrap_content\\\"\\n        android:layout_centerHorizontal=\\\"true\\\"\\n        android:layout_centerVertical=\\\"true\\\"\\n        android:text=\\\"Android Text: Hello World!\\\" />\\n\\n    <ImageView\\n        android:layout_width=\\\"200px\\\"\\n        android:layout_height=\\\"200px\\\"\\n        android:layout_centerHorizontal=\\\"true\\\"\\n        android:src=\\\"@drawable/icons_admin\\\"\\n        android:layout_above=\\\"@id/custom_title\\\"/>\\n\\n    <Button\\n        android:id=\\\"@+id/button1\\\"\\n        android:layout_width=\\\"match_parent\\\"\\n        android:layout_height=\\\"wrap_content\\\"\\n        android:layout_centerHorizontal=\\\"true\\\"\\n        android:layout_below=\\\"@id/custom_title\\\"\\n        android:text=\\\"Android Button\\\"\\n        android:clickable=\\\"true\\\"/>\\n</RelativeLayout>\\n\",\n      \"language\": \"xml\",\n      \"name\": \"activity_main.xml\"\n    }\n  ]\n}\n[/block]\nNext, we'll perform the usual Android view setup of inflating our views and adding click listeners to them. In the example below, we will be changing the text when the user clicks on the Android button.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// Basic Android view setting (Create / bind your layouts here).\\nprivate View inflateAndroidView(){\\n    final View androidViews = getLayoutInflater().inflate(R.layout.activity_main, null);\\n    final TextView text = (TextView) androidViews.findViewById(R.id.custom_title);\\n    final Button button = (Button) androidViews.findViewById(R.id.button1);\\n    button.setOnClickListener(new View.OnClickListener() {\\n        @Override\\n        public void onClick(View v) {\\n            text.setText(\\\"Android Text: Clicked on Button!\\\");\\n        }\\n    });\\n    \\n    return androidViews;\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nFinally, upon renderer startup, we'll create our AndroidViewTexture, and attach our desired android views to it. We then apply this texture onto a surface and set the appropriate click listener such that users can interact with this view.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// Basic Android view setting (Create / bind your layouts here).\\nfinal View androidViews = inflateAndroidView();\\n\\n// Create a Viro AndroidViewTexture that represents our Views.\\nint pxWidth = 400;\\nint pxHeight = 822;\\nboolean isAccelerated = true;\\nAndroidViewTexture androidTexture = new AndroidViewTexture(mViroView, pxWidth, pxHeight, isAccelerated);\\nandroidTexture.attachView(androidViews);\\n\\n// Set the Texture to be used on our surface in 3D.\\nfinal Material material = new Material();\\nmaterial.setDiffuseTexture(androidTexture);\\n\\nQuad surface = new Quad(0.5f, 1f);\\nsurface.setMaterials(Arrays.asList(material));\\nNode surfaceNode = new Node();\\nsurfaceNode.setGeometry(surface);\\nsurfaceNode.setPosition(new Vector(0,0,-1));\\n\\n// Add clicklisteners to \\nsurfaceNode.setClickListener(androidTexture.getClickListenerWithQuad(surface));\\n\\n// Add the Surface to the scene.\\nmARScene.getRootNode().addChildNode(surfaceNode);\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nFinal Result:\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/5542f08-ezgif-5-13c9f2c684.gif\",\n        \"ezgif-5-13c9f2c684.gif\",\n        300,\n        617,\n        \"#343f4a\"\n      ]\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"title\": \"Known Limitations\",\n  \"body\": \"AndroidViewTextures only supports rendering basic Android View Controls in AR/VR. However, rendering Android Views that contains GLSurfaces or dynamic web views are not supported.\"\n}\n[/block]","excerpt":"","slug":"android-ui-in-3d","type":"basic","title":"Android Views in 3D"}

Android Views in 3D


ViroCore also enables developers to enhance their applications with interactive 2D Android views by rendering them in AR or VR! As such, developers can easily build their familiar XML layouts, Android Buttons and ListViews in Android, and ViroCore will do all the heavy-lifting of rendering those views in a 3D space. [block:image] { "images": [ { "image": [ "https://files.readme.io/a2bbdd3-ezgif-5-4934b4cfe2.gif", "ezgif-5-4934b4cfe2.gif", 480, 270, "#9bb050" ] } ] } [/block] To use 2D Android UI components in VR: 1. Create an AndroidViewTexture with a width and height size representing your Android layout. 2. Attach your Android layout view to the AndroidViewTexture. 3. Apply this texture to any geometry you want to render the Android views on - a Cube, a Sphere or a Quad! You can also refer to the example code below for a more detailed implementation walkthrough. ### View Sizes and Hardware Support When constructing an AndroidViewTexture in Virocore, it is important to consider view sizing limitations - that is, the area of your Android View layout that is rendered before it gets clipped. By default, ViroCore's AndroidViewTexture will render AndroidViews onto a Viro Texture without any hardware acceleration. This ensures maximum compatibility with most Android OS versions, while guaranteeing correct view interaction behavior. However, there is a **memory size limitation** with this approach, resulting in potentially smaller viable width and height sizes for the texture. To check if your desired view size works with software acceleration, you can also use the helper function **AndroidViewTexture.suuportsSoftwareSurfaceOfSize(Context, desiredWidth, desiredHeight)** Else, developers can also force the AndroidViewTexture to use hardware acceleration, enabling **larger texture sizes**. Although most basic Android controls function properly with this mode, it is important to note that some views may have unpredictable behavior, particularly when handling user interaction. [block:callout] { "type": "warning", "body": "Note: If you see a black texture instead of your expected Android views, this is most likely an indication that the set width and height pixel values on your AndroidViewTexture has exceeded the device's capability to render them." } [/block] ### Events and Interactivity Users can also interact with and click on Android views that are attached to AndroidViewTextures in 3D (for example clicking on an Android Button with a VR reticle). To achieve this behavior, developers can use the **dispatchClickStateEvents API **on an AndroidViewTexture to map a Viro click event to a pixel coordinate within the bounds of the AndroidViewTexture's size. With that API, ViroCore then effectively plumbs through those click events to be triggered normally as if it were and Android click event being passed onto your Android views. Note that the mapping of any click event's position to an AndroidViewTexture's rectangular layout will be determined by the geometry shape that you have applied that texture to (however we have made it easy to do so for flat rectangular surfaces, as shown below). For convenience, developers can also use the AndroidViewtexture.getClickListenerWidhQuad(quad) helper function. This helper function helps construct a Viro ClickListener **that maps Viro clicks performed on the given quad to Android touch events**, which will then be automatically applied to the underlying view. This also takes into account the transformations that are applied to the quad, and as well as the size of the underlying texture. ### Example Code: In this example, we'll be attempting to render our Android layout shown below in AR. This layout will have an Android image, Android Text and an Android Button, all encapsulated within a RelativeLayout. [block:image] { "images": [ { "image": [ "https://files.readme.io/4d4c38c-button.png", "button.png", 1080, 2220, "#090909" ], "caption": "activity_main.xml" } ] } [/block] The representation of the above view in xml: [block:code] { "codes": [ { "code": "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<RelativeLayout\n xmlns:android=\"http://schemas.android.com/apk/res/android\"\n android:id=\"@+id/activity_main\"\n android:layout_width=\"match_parent\"\n android:layout_height=\"match_parent\"\n android:background=\"#99FFFFFF\">\n\n <TextView\n android:id=\"@+id/custom_title\"\n android:layout_width=\"wrap_content\"\n android:layout_height=\"wrap_content\"\n android:layout_centerHorizontal=\"true\"\n android:layout_centerVertical=\"true\"\n android:text=\"Android Text: Hello World!\" />\n\n <ImageView\n android:layout_width=\"200px\"\n android:layout_height=\"200px\"\n android:layout_centerHorizontal=\"true\"\n android:src=\"@drawable/icons_admin\"\n android:layout_above=\"@id/custom_title\"/>\n\n <Button\n android:id=\"@+id/button1\"\n android:layout_width=\"match_parent\"\n android:layout_height=\"wrap_content\"\n android:layout_centerHorizontal=\"true\"\n android:layout_below=\"@id/custom_title\"\n android:text=\"Android Button\"\n android:clickable=\"true\"/>\n</RelativeLayout>\n", "language": "xml", "name": "activity_main.xml" } ] } [/block] Next, we'll perform the usual Android view setup of inflating our views and adding click listeners to them. In the example below, we will be changing the text when the user clicks on the Android button. [block:code] { "codes": [ { "code": "// Basic Android view setting (Create / bind your layouts here).\nprivate View inflateAndroidView(){\n final View androidViews = getLayoutInflater().inflate(R.layout.activity_main, null);\n final TextView text = (TextView) androidViews.findViewById(R.id.custom_title);\n final Button button = (Button) androidViews.findViewById(R.id.button1);\n button.setOnClickListener(new View.OnClickListener() {\n @Override\n public void onClick(View v) {\n text.setText(\"Android Text: Clicked on Button!\");\n }\n });\n \n return androidViews;\n}", "language": "java" } ] } [/block] Finally, upon renderer startup, we'll create our AndroidViewTexture, and attach our desired android views to it. We then apply this texture onto a surface and set the appropriate click listener such that users can interact with this view. [block:code] { "codes": [ { "code": "// Basic Android view setting (Create / bind your layouts here).\nfinal View androidViews = inflateAndroidView();\n\n// Create a Viro AndroidViewTexture that represents our Views.\nint pxWidth = 400;\nint pxHeight = 822;\nboolean isAccelerated = true;\nAndroidViewTexture androidTexture = new AndroidViewTexture(mViroView, pxWidth, pxHeight, isAccelerated);\nandroidTexture.attachView(androidViews);\n\n// Set the Texture to be used on our surface in 3D.\nfinal Material material = new Material();\nmaterial.setDiffuseTexture(androidTexture);\n\nQuad surface = new Quad(0.5f, 1f);\nsurface.setMaterials(Arrays.asList(material));\nNode surfaceNode = new Node();\nsurfaceNode.setGeometry(surface);\nsurfaceNode.setPosition(new Vector(0,0,-1));\n\n// Add clicklisteners to \nsurfaceNode.setClickListener(androidTexture.getClickListenerWithQuad(surface));\n\n// Add the Surface to the scene.\nmARScene.getRootNode().addChildNode(surfaceNode);", "language": "java" } ] } [/block] Final Result: [block:image] { "images": [ { "image": [ "https://files.readme.io/5542f08-ezgif-5-13c9f2c684.gif", "ezgif-5-13c9f2c684.gif", 300, 617, "#343f4a" ] } ] } [/block] [block:callout] { "type": "warning", "title": "Known Limitations", "body": "AndroidViewTextures only supports rendering basic Android View Controls in AR/VR. However, rendering Android Views that contains GLSurfaces or dynamic web views are not supported." } [/block]