{"pageProps":{"editions":[{"mdPath":"/vercel/path0/content/editions/0124.md","num":124,"rawMd":"\n\nOne of the great things about [Appium 2.0](https://appium.io) is that you can find drivers to\nenable your automation testing across a wider variety of platforms than ever before; now including\neven media streaming platforms! In this article, we'll learn how to test Roku TV applications using\na relatively new Appium driver for Roku TV. I implemented this driver on behalf of HeadSpin, and\nit's fully open source, available on GitHub: [Appium for Roku\nTV](https://github.com/headspinio/appium-roku-driver). The full documentation for the driver is\navailable there at the README, but we'll be covering all the highlights in this article.\n\n### How the Appium Roku driver works\n\nTo perform automation well on a given platform, it's always worth doing a little research on how\napp development for the platform works, beyond finding an Appium driver for it. This helps you\nunderstand how the driver itself works and what its requirements and limitations are. The more you\nknow about iOS development, for example, the better use you'll be able to make of Appium's iOS\ndriver. The same goes for Roku! The best source for Roku developer information is [Roku's Developer\nPortal](https://developer.roku.com/en-ca/docs/developer-program/getting-started/roku-dev-prog.md).\nThis is the documentation I used to build the Appium Roku driver.\n\nOne thing to note right off the bat is that Roku apps are called \"channels\" (as in TV channels) in\nthe Roku documentation, so be aware of how this language is a bit different from most platforms.\n\nYou might also notice that Roku publishes something called the [Roku\nWebDriver](https://developer.roku.com/en-ca/docs/developer-program/dev-tools/automated-channel-testing/web-driver.md),\nwhich is a WebDriver server implementation for Roku automation, written in Go. Its purpose is very\nsimilar to the purpose of the Appium Roku driver---to allow for WebDriver-style automation of Roku\napps. So why did I bother to make an Appium Roku driver at all? When I looked into the Roku\nWebDriver implementation, it unfortunately did not follow standard W3C protocol conventions, nor\nwas it already Appium-compatible. Reading through the code, it became apparent that the Roku\nWebDriver server was a thin layer on top of another Roku developer tool called the [External\nControl\nProtocol](https://developer.roku.com/en-ca/docs/developer-program/dev-tools/external-control-api.md)\n(ECP). This is an API hosted on the Roku device itself which allows limited remote control and\ndebugging of Roku developer channels. Beyond this, I noticed that the Roku WebDriver did not have\ncertain features I thought were important, like the ability to find element objects and trigger\nthem by calling the `Click Element` command. Ultimately, it seemed like a better idea to create and\nmaintain an Appium driver based on the same protocol, so that's what we did!\n\nThe ECP takes care of most of how the Appium driver works, since it lets you do things like get\na dump of the UI element hierarchy or simulate remote control keypresses. But it doesn't let you do\nthings like install apps or take screenshots. Luckily, the Roku comes with a special developer mode\nwebsite that you can access in a web browser locally connected to the Roku device. This website\nlets you upload your development apps (called \"dev channels\"), and retrieve screenshots. The Appium\ndriver simply calls out to this webserver when it needs to perform these tasks as part of its work,\nso you don't have to do it manually.\n\n### Requirements and limitations of the Appium Roku driver\n\nBefore beginning an automation project for any platform, it's important to make sure you understand\nthe requirements and limitations of the platform, so you're not stuck in a dead end. This is\ndefinitely true for Roku test automation as well. Here are some of the requirements you'll need to\nkeep in mind if you want Roku testing to be successful:\n\n1. You will need a physical Roku device for testing. There is no such thing as a Roku emulator,\n unfortunately.\n1. Your Roku needs to be on the same network as your Appium server (since the Appium server will\n make network calls to the Roku).\n1. Your Roku needs to be in [Developer\n Mode](https://developer.roku.com/en-ca/docs/developer-program/getting-started/developer-setup.md)\n so that the ECP server and Dev Mode server are up and running.\n1. You need to have access to your Roku channel as a `.zip` file in the format expected by the Roku\n channel sideloading (installation) service. In other words, it should be an archive of your\n source code, not a compiled version of the app.\n1. And of course, you should have a functional Appium 2.0 server environment, ideally updated to\n the latest and greatest!\n\nHere are some limitations of the driver to keep in mind:\n\n1. You cannot automate channels that you do not have the source code to (i.e., your dev channel).\n (Technically, you can still use the driver's remote control emulation to do anything you want on\n the device, but this will be driving the device \"blind\"---you'll have no way to check the UI\n hierarchy or grab a screenshot). I.e., the `Take Screenshot` and `Get Page Source` commands only\n work if the dev channel is active.\n1. Roku provides a way to see the UI element hierarchy, but no way to work with specific UI\n elements directly. The only way to interact the device is via the remote control. The Appium\n driver does however provide a special algorithm for emulating the `Click Element` command which\n will be highlighted below. It also has a convenience method for `Send Keys`, that simply sends\n keystrokes to the Roku as if a user were doing so with the remote. All this to say: you will be\n able to run the `Find Element` command, which is certainly useful for waiting for app state, but\n in general there is no way to interact with the element you have found. To trigger elements\n you'll need to use the driver's remote control methods or one of the few convenience methods\n mentioned above.\n1. Only one dev channel may be sideloaded at any given time. When you (or the driver) sideloads\n a dev channel, it will be given the channel ID of `dev`. So even though your app will have its\n own official channel ID used for publishing, the ID of the channel for automation purposes will\n basically always be `dev`.\n1. When you take a screenshot of a screen which contains video content, that video content will be\n shown as a black rectangle. This is because Roku does not allow DRMed video content to leave the\n device except to authorized playback machines (like TVs).\n\n### Installing the Appium Roku driver\n\nWith the Appium 2.0 extension CLI, installing the driver is as easy as running:\n\n```bash\nappium driver install --source=npm @headspinio/appium-roku-driver\n```\n\nThis will get you the latest version of the Roku driver. If you would like to upgrade at any point\nlater, you can simply run:\n\n```bash\nappium driver update roku\n```\n\n### Starting Roku app test automation sessions\n\nTesting your Roku TV apps using this driver involves using the following required capabilities in\nwhichever Appium client you're most comfortable with:\n\n|Capability|Description|\n|----------|-----------|\n|`platformName`|Should always be `Roku`|\n|`appium:automationName`|Should always be `Roku`|\n|`appium:app`|An absolute path to the zip file of the dev channel you want to install. If not included, a session will simply be started on the home screen. You can also just set it to `dev` if the dev channel is already installed.\n|`appium:rokuHost`|The host name or IP of the Roku device on the network|\n|`appium:rokuEcpPort`|The ECP port on the Roku device (usually `8060`)|\n|`appium:rokuWebPort`|The dev web interface port on the Roku device (usually `80`)|\n|`appium:rokuUser`|The username you selected when turning on dev mode|\n|`appium:rokuPass`|The password you selected when turning on dev mode|\n|`appium:rokuHeaderHost`|The IP of the Roku device on *its* local network (should usually the same as `rokuHost` unless you are tunneling or connecting via DNS)|\n\nThere is also a special capability that determines how long the driver will wait in between remote\nkey presses. This can be useful if the automated key presses are happening too quickly for your app\nto react to them. If you find yourself in this position, simply set the `appium:keyCooldown`\ncapability to the number of milliseconds you want to wait in between key presses.\n\n### Using a session to test Roku TV apps\n\nOnce you have a session going in your Appium client, you'll navigate your app and implement your\nautomation test for Roku using primarily the following commands (note that the way to write these\ncommands differs based on the client library you're using; you can refer to the client libray\ndocumentation for specific examples):\n\n|Command|Description|\n|-------|-----------|\n|`Get Page Source`|Returns an XML document representing the current UI hierarchy. You should use this document to come up with XPath queries for elements you want to interact with.|\n|`Get Screenshot`|Returns a PNG image representing the current screen.|\n|`roku: pressKey`|This is an [Execute Method](https://appium.io/docs/en/2.0/guides/execute-methods/) that allows you to trigger remote control key presses, like the home button or the play/pause buttons. This method takes a single parameter named `key`, whose value should be one of the [supported key values](https://developer.roku.com/en-ca/docs/developer-program/debugging/external-control-api.md#keypress-key-values) from the Roku documentation. Using this method, you'll be able to do anything a user would be able to do!|\n|`Find Element` / `Find Elements`|Using an XPath query as your selector, get a reference to an element found in the UI hierarchy.|\n|`Click Element`|Once you have an element, you can call `.click()` on it. When you do this, the Roku driver will go through a special algorithm to attempt to automatically focus the element you found and hit the `Enter` button to trigger its behaviour. Hopefully it will save you from having to map out the precise remote key presses required to navigate to the element.|\n|`Send Keys`|If you need to type text into an input field, first make sure the input field is active and ready to receive text input. Then call the send keys / set value command (whatever it's called in your Appium client), and the driver will use the special remote control API to send literal characters instead of remote key presses.|\n\nThere are a number of other commands available to help you in your Roku TV test automation (which\nyou can learn more about at the driver documentation), including:\n\n- Install and remove apps\n- Launch apps\n- List available apps\n- Get information about the device\n\nThat's it! This guide should be all you need to get started with automation testing for Roku. Now\ngo forth and test your streaming apps on Roku TV devices! Remember, the project is open source, so\nif you think you've found a bug, please report it at the [issue\ntracker](https://github.com/headspinio/appium-roku-driver/issues), and of course, contributions are\nalso welcome.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Using Appium for Automated Testing of Roku TV Apps","shortDesc":"One of the drivers available in the Appium 2.0 ecosystem is the Appium Roku driver, which enables you to test Roku TV applications.","canonicalRef":"https://www.headspin.io/blog/using-appium-for-automated-testing-of-roku-tv-apps","liveAt":"2023-04-27 14:00"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/124-using-appium-for-automated-testing-of-roku-tv-apps","slug":"124-using-appium-for-automated-testing-of-roku-tv-apps","path":"/editions/124-using-appium-for-automated-testing-of-roku-tv-apps","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0123.md","num":123,"rawMd":"\n\nAppium 2.0 is almost here! Most of the foundational work has been done, and it has been in beta for some time now. If you haven't yet gotten the Appium 2.0 beta, you can read the Appium Pro article on [installing Appium 2.0 and Appium 2.0 drivers](https://appiumpro.com/editions/122-installing-appium-20-and-the-driver-and-plugins-cli) to get started. Apart from the differences in how you make platform drivers available to your Appium install, there will be a number of other (mostly minor) breaking changes you need to be aware of once you start migrating your Appium 1.x scripts to Appium 2.x.\n\nThis article is the first in an ongoing series that will attempt to walk you through all these changes. The topic of discussion for this article is capability prefixes. \"Capabilities\" (formerly known as \"desired capabilities\") are the parameters you use to start an Appium session. They come in key/value pairs, (for example, `platformName` which allows you to specify which platform you want to automate, whether that is iOS, Android, Mac, Windows, etc...). Depending on your Appium client, you define capabilities in a variety of ways, either using capability enum objects or strings (as in Java) or by constructing a dict (as in Python).\n\nWith Appium 2.0, the Appium server will enforce strict compability with the [W3C WebDriver specification](https://github.com/jlipps/simple-wd-spec) when it comes to Capabilities. The spec defines 9 official capabilities, many of which aren't used very frequently in Appium. The standard, official capabilities you're probably most familiar with could be the following:\n\n* `browserName`\n* `platformName`\n* `timeouts`\n\n(You can check out the others at the [full list of official capabilities](https://github.com/jlipps/simple-wd-spec#capabilities)). When it comes to these standard capabilities, there are no breaking changes. But every other capability that you use with Appium tests (like `deviceName` or `automationName` or `udid` or `app`), are _non_-standard, from the perspective of the W3C spec. Going forward, Appium will reject these capabilities. So how can we keep sending Appium the information they are designed to send? Luckily, the W3C spec defines a way for non-standard capabilities to be accepted as valid. These capabilities must include a so-called \"vendor prefix\", which looks like a string ending in a colon. If we were to take the `automationName` capability as an example, we could turn it into a valid W3C capability by adding the `appium:` prefix:\n\n```text\nappium:automationName\n```\n\nWith that background in mind, let's explore 3 different strategies for updating your tests to use these vendor prefixes appropriately, in order to avoid test failures during new session creation.\n\n### Strategy #1: Manually update your capabilities\n\nThe first strategy is perhaps the best long term strategy, but it involves the most work. Just go into your test code and update every instance of a non-official capability so that it has the `appium:` prefix!\n\n### Strategy #2: Use an updated Appium client\n\nThe Appium team is obviously aware that it's not pleasant to have to find and change every instance of a capability everywhere in a testsuite. So we've also been hard at work updating our client libraries so that, if the library detects a capability that would be invalid without an `appium: ` vendor prefix, it will add that prefix automatically.\n\nWhat this means is that if you're simply using the most recent version of the Appium client library, the vendor prefix addition will happen for you under the hood. This is probably the simplest path forward at the present time, since it involves no work on your part other than updating your client, which is probably a good idea independently.\n\n### Strategy #3: Use the `relaxed-caps` plugin\n\nIf you can't upgrade your client, or are using a Selenium client, don't give up hope! There's still a way to have Appium give you a bit of a break. While the Appium server itself will only accept valid W3C capabilities, I've written a plugin that adjusts Appium's behavior to be a little more lenient. When this plugin is active, it will accept the old-style, non-prefixed capabilities (well, secretly, what it does is just add them before you before the capabilities get to the new session creation function). The net result is that once again your tests can stay the same.\n\nThis plugin is called the [relaxed-caps](https://github.com/appium/appium-plugins/tree/master/packages/relaxed-caps) plugin, and it's one of the officially-supported Appium 2.0 plugins. You can see it, for example, if you run `appium plugin list`. And you can install it by running:\n\n```bash\nappium plugin install relaxed-caps\n```\n\nThen of course you'll need to activate it when you start your Appium server, to make sure you know what you're allowing in terms of plugins and session modification:\n\n```bash\nappium --use-plugins=relaxed-caps\n````\n\nSo this concludes our first discussion of breaking changes in Appium 2.0. I hope that one of these 3 strategies will work well for you, and at least give you the time to update your capabilities to use the appropriate vendor prefix!\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Migrating to Appium 2.0, Part 1: Capability Prefixes","shortDesc":"Appium 2.0 will soon be the only supported version of Appium. What do you need to know about updating your test scripts so that they keep working when you upgrade? This is the first part of an open-ended series detailing the breaking changes you might encounter when migrating to Appium 2.0","liveAt":"2021-06-17 14:00","canonicalRef":"https://www.headspin.io/blog/migrating-to-appium-2-0-part-1-capability-prefixes"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/123-migrating-to-appium-20-part-1-capability-prefixes","slug":"123-migrating-to-appium-20-part-1-capability-prefixes","path":"/editions/123-migrating-to-appium-20-part-1-capability-prefixes","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0122.md","num":122,"rawMd":"\n\n> If you've been around the Appium world for a while, you've probably heard that Appium 2.0 has been \"coming soon\" for a very long time! I'm happy to report that work on it has been progressing well, and Appium 2.0 is now ready to use as a beta!\n\n### Appium 2.0's vision\n\nBefore we get into the details of installing and running Appium 2.0, it's worth mentioning some of the core goals for this next major revision of Appium:\n\n* **Decouple the drivers!** Appium's platform drivers (the XCUITest driver, UiAutomator2 driver, Espresso driver, etc...) have very little in common with one another. They really ought to be developed as independent projects that implement the same interface and can be used equivalently with the Appium server. With Appium 2.0, the code for these drivers will no longer be bundled with the main Appium server. This decreases the size of an Appium install dramatically, and makes it so that you don't need to install drivers that you don't need to use. It also makes it possible to freely update drivers independently of Appium and of one another, so that you can get the latest changes for one driver while sticking with a known stable version of another driver, for example.\n* **Create a driver ecosystem.** Once the drivers are decoupled from Appium, it's quite an obvious question to ask: what's special about these drivers, anyway? Why couldn't anyone else create a driver for their own platform? Well with Appium 2.0, they can, and they should! By using any existing Appium drivers as a template, anyone can create their own custom drivers with a minimum of extra code. All of these custom drivers can then be installed by any Appium user (or custom drivers could be private, or sold, or whatever you can dream of).\n* **Create a plugin ecosystem.** In addition to drivers, it's become clear that there are a huge variety of use cases for Appium, which involve the use of special commands or special ways of altering the behavior of Appium for specific commands. Some good examples of this would be the [Find Element by Image API](https://appiumpro.com/editions/32-finding-elements-by-image-part-1) or the [Appium + Test AI Classifier](https://appiumpro.com/editions/101-ai-for-appium--and-selenium). Not every automation use case requires these features, but the code and dependencies that support these features are included with every Appium install. It would be better to be able to install these features as independent plugins. And it would be even better for anyone in the world to be able to easily create Appium plugins that can implement new commands, or alter the behavior of existing commands! Using the same model as our driver ecosystem, anyone can create plugins like these and easily share them with the world of Appium users.\n* **Miscellaneous standardization**. There has been a lot of deferred work on Appium that kept getting pushed off because it could introduce a breaking change into the API. We're going to take the opportunity to make those changes now, and keep bringing Appium into the future.\n\n### Installing Appium 2.0\n\nAt the moment, Appium 2.0 is not the main line of Appium development, so it cannot be installed with a simple `npm install -g appium`. Instead, Appium 2.0 beta versions will be available with a special NPM tag `next`, so you can install it on any platform using NPM as follows:\n\n```\nnpm install -g appium@next\n```\n\nThat's it!\n\n### Installing Appium drivers\n\nAt this point, after installing Appium 2.x for the first time, if you run the server, you'll get a line in the logs that looks something like this:\n\n> [Appium] No drivers have been installed. Use the \"appium driver\" command to install the one(s) you want to use.\n\n(And you'll get the same message with respect to plugins). What this is telling us is that you need to have Appium install a driver before you run any tests. That's because no drivers are included by default with Appium 2.x. Instead, you tell Appium which drivers you care to use. Those drivers can then be removed or updated as necessary, all without having to change your version of Appium! So let's take a look at the commands you could use to install, say, the XCUITest and UiAutomator2 drivers:\n\n```bash\nappium driver install xcuitest\nappium driver install uiautomator2\n```\n\nSo essentially, there is now a new set of \"subcommands\" for the main `appium` program you run from the CLI. The subcommand we're looking at here is the `driver` subcommand, which has its *own* set of subcommands! The one we just saw here is the `install` subcommand for the driver CLI. In the normal case, it takes a single parameter which is the name of the driver you want to install. How did I know which strings to use (`xcuitest` and `uiautomator2`)? Well, Appium knows about some \"official\" drivers which you can install just by name. To get a list of all these \"official\" drivers, you can run `appium driver list` (see below).\n\nBefore we talk about `driver list`, let's explore the `driver install` command a bit more. The full spec for the command looks like this:\n\n```bash\nappium driver install --source= --package= \n```\n\n#### The `--source` option\n\nThe `--source` option is not required, but can be included to tell the `driver install` command where to find the driver you want to install, if you're not installing one of Appium's \"official\" drivers. Basically it tells Appium how it should treat the `installSpec` you include, and opens up the possibility of installing drivers from practically anywhere! There are 4 options here:\n\n|Source|Meaning|\n|----|----|\n|`npm`|Install a driver as an NPM package|\n|`github`|Install a driver from a GitHub repo|\n|`git`|Install a driver from an arbitrary Git repo|\n|`local`|Install a driver from a local path on the filesystem|\n\nWe'll talk about what each of these mean for `installSpec` in a moment.\n\n#### The `--package` option\n\nIf you use the `--source` option, and if your source is something other than `npm`, then you must also include this `--package` option, and its value needs to be the package name of the driver module. The driver doesn't actually need to be *published* to NPM or anything, but since every Appium driver is a Node.js package, it will have a package name which Appium will need to know in order to find the driver when it is downloaded and installed.\n\n#### The `installSpec` parameter\n\nThe only required parameter is what we call the \"install spec\". It can be in different formats depending on your source. The following table illustrates the possibilities:\n\n|Source|Install Spec Format|Example|\n|--|--|--|\n|(no source)|The name of an official driver, as found in `appium driver list`|`xcuitest`|\n|`npm`|The name of an NPM package plus any valid NPM restrictions or tags(which could be installed|`customDriver@1.0.2`|\n|`github`|The GitHub org and repo as `/`|`appium/appium-xcuitest-driver`|\n|`git`|The fully qualified Git URL of the repo|`https://github.com/appium/appium-xcuitest-driver.git`|\n|`local`|The path to the driver on the disk|`/usr/local/drivers/custom-driver`|\n\n### Other driver CLI commands\n\nA few times we've mentioned the importance of `appium driver list`. What does this command do? It tells you which drivers you have installed, and which official drivers you *could* install that you haven't yet! For example, this is the output when I've installed only the XCUITest driver:\n\n```text\n✔ Listing available drivers\n- xcuitest@3.31.5 [installed (NPM)]\n- uiautomator2 [not installed]\n- youiengine [not installed]\n- windows [not installed]\n- mac [not installed]\n- espresso [not installed]\n- tizen [not installed]\n- flutter [not installed]\n```\n\nAny drivers that are installed will display their version, and which source was involved in installing the drivers. You can also use `driver list` to check and see if any drivers have any updates available:\n\n```bash\nappium driver list --updates\n```\n\nIf an update is available, you'll see that in the output. You can then update a specific driver as follows:\n\n```bash\nappium driver update \n```\n\n(where `driverName` is the name of the driver as printed in the output of `appium driver list`.) Note that by default Appium will not let you update a driver across a major version boundary, to keep you safe from breaking changes. Once you've done your due diligence on any breaking changes with the next major version of a driver, you can proceed with the upgrade as follows:\n\n```bash\nappium driver update --unsafe\n```\n\nAlso, you can update *all* installed drivers in one go, using the special driver name `installed`:\n\n```bash\nappium driver update installed\n```\n\nAnd that's about it for the driver CLI! Note that you can of course also uninstall particular drivers if you no longer want them around (this is also how you'd update non-NPM based drivers, using a combination of uninstall and then install again):\n\n```\nappium driver uninstall \n```\n\n### Setting the driver repository location\n\nOne question you might have had is: where does Appium install drivers when you use the driver CLI? By default, Appium creates a hidden directory called `.appium` inside your user home directory. But if you'd like to use a different directory, or if you want to keep multiple driver repositories around, you can adjust this path by using one of these three command line options (which are all aliases of the same behavior):\n\n* `-ah`\n* `--home`\n* `--appium-home`\n\nFor example, `appium -ah /path/to/custom/appium/home driver install xcuitest`.\n\n### More to come...\n\nAll we talked about in this article was installing Appium 2.0 and using the driver CLI. But there's a lot more to Appium 2.0! In future articles we'll look at some of the breaking changes that are coming, and most importantly how to leverage the plugin CLI. But for now, you can feel free to give Appium 2.0 a whirl, and see how it works for you!\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Installing Appium 2.0 and the Driver and Plugins CLI","shortDesc":"In this article we take a first look at Appium 2.0: its goals, its installation, and the ins and outs of the very important new 'driver' command line interface.","newsletterTopNote":"By the way, sorry for the radio silence from Appium Pro! I've been hard at work on my massive Appium and Selenium Fundamentals course, which is now complete and nearing public release. You'll hear more from me in a little bit about that. In the meantime, hope you enjoy this new article!","liveAt":"2020-12-02 14:00","canonicalRef":"https://www.headspin.io/blog/installing-appium-2-0-and-the-driver-and-plugins-cli"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/122-installing-appium-20-and-the-driver-and-plugins-cli","slug":"122-installing-appium-20-and-the-driver-and-plugins-cli","path":"/editions/122-installing-appium-20-and-the-driver-and-plugins-cli","news":{"rawMd":"I thought many of you might like to know that Appium Pro sponsor HeadSpin now has a manual testing platform called [HeadSpin Local](https://www.headspin.io/platform/local/?utm_source=appiumpro&utm_medium=email&utm_campaign=local), which offers testing on real devices around the world.\n\nThe idea here will probably be very obvious to you; you can simply select a device (or multiple) from a diverse pool of iOS and Android devices and control those devices via a remote workbench that has all your tools, including device logs and screenshots that you can save and share with your team. Remote automation via Appium will of course be released down the line as a forthcoming product, but I've found that just being able to open up a specific device in a specific geographical location is pretty useful for a sanity check or a bug repro.\n\nAnyway, right now there's a 25% off special for the yearly version of the plan, so if you’re curious, visit our page to [get started](https://www.headspin.io/platform/local/?utm_source=appiumpro&utm_medium=email&utm_campaign=local)!\n","num":122,"mdPath":"/vercel/path0/content/news/0122.md"},"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0121.md","num":121,"rawMd":"\n\n> Appium Pro is usually known for Appium-related automation tutorials and advanced techniques. But sometimes it's good to step back and take a look at the bigger picture of how these techniques are used, which is most frequently in the context of \"testing\". In my forthcoming course on HeadSpin University, [Appium and Selenium Fundamentals](https://ui.headspin.io/university/courses?utm_source=appiumpro&utm_medium=referral&utm_campaign=hs_university), I talk a lot about testing, and what it is, before diving into any automation training. I think it's always useful to consider our practices from a philosophical perspective before just plowing ahead, and so this article is a snippet of the course which has to do with precisely this question: what is testing?\n\n

\n \"Ancient\n

\n\nI think we would all have a fairly good intuitive idea of what testing is. It's making sure something works, right? Indeed! But as a former linguist, I think it'd be great to dive a bit into the etymology of testing, the background and history of the word itself, to get a sense of the roots of the concept.\n\n### The Etymology of Testing\n\nThe English word \"test\" originally came from a Latin word *testum*, which referred a kind of container or pot made of earth. These were pots that were designed to hold very hot metal. People wanted to heat metal up for a number of reasons, but one was to determine whether there were any impurities in the metal.\n\nSo, by association, the pot itself became tied up with the act of checking the quality of the metal, and so now we can say that to put something to the test, i.e., to heat it up in the earthen pot called a \"testum\", is to determine the quality of whatever that thing is! You might also have heard someone talk about a \"crucible\", or maybe heard the expression \"going through the crucible\". This is another word that has a very similar history. A \"crucible\" is another kind of container for heated metal! To take our analogy a bit deeper, we can say that \"testing\" is the process of applying some kind of energy in some kind of container to determine the quality of something we care about.\n\n### Testing Software\n\nNow, what *we* care about is software. Our job as testers is to get some kind of indication of the quality of the software that sits in front of us. Maybe we developed it ourselves, or maybe someone else did. How do we even start thinking about the process of testing software? There are a huge number of methods and tools that we can use in the process. But one simple way of thinking about it is realizing that our responsibility is to design and bring about the specific conditions (like heating a pot) that will reveal the quality of the software. One common way of bringing about these conditions is to use the software in its normal modes, as a normal user. The software is meant to run in certain conditions, and we can simply run it in those conditions, as any normal user of the software would, to determine whether it works the way it's supposed to in those conditions.\n\nWe could *also* try to imagine conditions for the software that its developers might not have thought about. These are often called \"edge cases\". The trouble with software is that there can be a near infinite number of conditions a piece of software runs in, and a near infinite number of situations or states it can get itself into, based on user input or other factors. When developers write software, they usually assume a fairly simple and minimal set of conditions. It is the job of a tester to design crucibles, or test scenarios, that show how the software behaves in all kinds of situations.\n\nObviously, if there are an infinite number of situations that software can get run in, it doesn't make sense to try and test every possible case. So testing becomes a real mix of engineering and creative, out-of-the-box thinking, just to figure out *what* to test. And that is only the beginning! Testers don't have an infinite amount of time to evaluate each potential release of software. Sometimes software gets released multiple times a day! Choices have to be made about what's most important to test, and which additional tools and strategies to adopt to make sure that as much as possible can be tested.\n\nImagine if you were an ancient Roman tester, and people brought their metal to you to heat up and determine its quality. Once the metal starts piling up faster than you can gather logs and heat up your pots, you've got a problem, and you need to figure out not just new and clever methods for determining the quality of the metal, but also the mechanics of how to scale your testing practice. Maybe you buy more pots, or invest in technology that starts fires faster. Maybe you get super advanced and figure out that by mixing certain reagents in with the metal, you can get an instant read on one aspect of the quality of that metal! Anyway, I don't want to push this analogy too far. But this is very much the situation encountered by testers. Almost all testing of software nowadays is done in a context that requires great breadth, depth, speed, and scale. It's an impossible set of requirements, unless testers adopt some of the same modes of thinking that have propelled software development to huge levels of scale.\n\nAnd yes, by that I mean automation, which is a conversation for another time. The main takeaway for now is this: testing is a practice which uses creativity and technology to build and execute special crucibles for software, with the goal of determining the quality of the software to as accurate a degree as possible!\n\n## Kinds of Testing\n\nSo, we defined testing as follows: the act of defining, building, and executing processes that determine application quality. This is a good definition, but it's pretty general. We can get a lot more specific about the various types and subtypes of testing, so we know what kinds of options we have when it comes to defining and building these test processes.\n\nThe way I like to think about types of testing is as a 3-dimensional matrix. Each dimension corresponds to one facet or characteristic of a test. Here are the three dimensions that I see:\n\n1. Test *targets*. A test target is basically the thing you want to test! We might call it the \"object under test\" to be general. In software testing, it's usually called the \"App Under Test\", sometimes written AUT. But target means more than saying what app you're testing. Usually we're only responsible for testing one app anyway, so the App Under Test is kind of a given. What I mean by \"target\" is, what *component* or *layer* of the app is being tested. When we develop software, the software usually has a number of distinct components, and a number of distinct levels. We can consider the levels independently, and even test them independently if we want. One component or level of an application could be an individual code function. Users don't execute individual code functions---that happens behind the scenes, so to speak. But we could test a specific function in isolation! Another layer could be the layer of an application's API. Again, users don't typically try to access an app's backend API. That's something that the app itself would do in order to provide the appropriate information to the user. Yet another layer could be the user interface or UI of the app. This is the part that a user actually interacts with! And we could test all of these different layers. We could test individual units of code. We could test APIs. We could test integrations between various backend services. And we could test the UI. That's what I mean by \"target\"---the test target is the answer to the question, what aspect of the app itself are you trying to test?\n\n2. Test *types*. Now here I mean something very specific by the word \"type\". In this sense, I mean, what aspect of the app's *quality* are you trying to test? Apps can have good or poor quality on a number of different dimensions. One app might be fast but perform calculations incorrectly. So we would say that it has good performance but poor functionality. Another app might have good performance and functionality, but be hard for people who are hard of hearing to use. Then we would say that app has poor accessibility. Performance, functionality, and accessibility are all aspects of the quality of an app. As apps become more and more important in our lives, we keep coming up with more and more aspects of app quality that become important! If we want to test all of these categories, I would say we want to implement a number of different test *types*. So that is what I mean by \"type\".\n\n3. Test *methods*. This category is a bit more straightforward. Basically, method just refers to the procedure used to implement the test. How will the test be performed? That's the method. What are some methods? Well, what we call \"manual testing\" is one method. Manual testing is simply the act of using an app the way a user would, or exercising an app component in a one-off way. In other words, it's done by hand, by a human being. You can manually test units of code. You can manually test APIs. And of course, the most common instance of manual testing is manual testing of app UIs. But manual testing isn't the only method. We can also write software to test our app under test. In this case, our tests are considered to be using the *automated* method.\n\nSo this is the structure I use to help give us an understanding of the various types of tests. These are three dimensions, meaning that in principle the test targets are independent from the test types, and they're both independent from test methods. In practice, not every possible combination of those three dimensions ends up making sense, but a lot do. Let's look at some examples!\n\n### Unit Testing\n\nThe name \"unit testing\" refers to the idea that we are testing a specific unit of code. Unit testing can be described using the different test dimensions in this way: the \"target\" of a unit test is typically a single function in the codebase. The \"type\" is usually understood to be functionality. Usually when writing unit tests, we are making sure that the specific bit of code in question functions appropriately, in terms of its inputs and outputs. We *could* run other types of tests on single code functions, for example we could test their performance. But usually unit tests are written to test functionality. And the \"method\" for a unit test is usually automation.\n\n### UI Testing\n\nUI Testing goes under many different names. Sometimes it's called \"end-to-end\" testing. Sometimes it's just called \"functional testing,\" and some people even refer to UI tests when they talk about \"integration tests.\" I think most of these other names are a bit confusing, so I prefer to use the term \"UI Test.\" This makes clear what the target of our testing is: it's the UI of an application. As we talked about earlier, an app has many different layers, but users typically only interact with the UI. When you are doing UI testing, you are running tests from the perspective of the user! Now, \"UI\" just refers to the target of testing, and it doesn't specify the type or method involved. Typically, if we don't say anything else, UI testing is assumed to be functional in nature. That is, we are testing the UI's functionality. And of course, we can do this using different methods, whether manual or automated.\n\n### Integration Testing\n\nAnother type of testing which is commonly discussed is 'integration testing'. Sometimes, when people say \"integration testing\" they might mean \"UI testing\" so it's good to make sure everyone's working with the same definition. When *I* say integration testing, what I mean is testing the communication, or \"integration\", between two different services. Let's go back to the 3-dimensional framework we've been using to unpack this a little bit. In terms of a target or targets, integration testing is focused on two components of a larger system. In the world of modern software development, the most obvious candidate for these components would be a set of microservices that all communicate in order to power an application. But the components don't necessarily need to be free-standing services; they could also just be two chunks of a much larger monolithic codebase, which are responsible for communicating in some way. The test \"type\" I mean when I talk about \"integration testing\" is usually \"functional\". We could do performance integration testing, but it's usually assumed that we're just talking about making sure the integration works at all. Finally, the test \"mode\" can be manual or automated, just like any other form of testing we've discussed, but this kind of testing is almost always automated.\n\nA good example of integration testing would be asserting that two services communicate with one another correctly, even if the entire stack is not in the picture. Maybe you have one service that reads customer orders from a task queue, does some processing on them, and then sends certain orders to a system that is responsible for physical order fulfilment (in other words, shipping), and sends other orders to a system that is responsible for digital order fulfilment (in other words, e-mailing digital products or something like that). You could image setting up a test environment that just runs these few services, and has no frontend, and no actual real database or task queue. Fake orders can be created in a simple in-memory queue, and orders received by upstream services can simply be included in a correctness assertion, but not actually acted on. This would be an example of an integration test.\n\n### Conclusion\n\nThere are lots of other kinds of testing out there, but I hope that you remember the three dimensions we can use to define any type of testing: the test *target*, the test *type*, and the test *mode*. The target specifies what thing we're trying to ensure quality for. The type specifies what sort of quality we care about (speed, correctness, visual appeal, etc...). The mode specifies how we go about the test (do we use automated means, or manual means, or some specific subcategory of those?). Take a moment and think about the various tests you've written in your career, and try to map them on this schema.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"What Is Testing?","shortDesc":"On Appium Pro we talk a lot about specific automation techniques, but we don't often talk about the philosophy or the art of testing. Let's do that now, with a thoughtful introduction to what testing itself is.","liveAt":"2020-09-02 00:00","canonicalRef":"https://www.headspin.io/blog/what-is-testing"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/121-what-is-testing","slug":"121-what-is-testing","path":"/editions/121-what-is-testing","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0119.md","num":119,"rawMd":"\nWhen I was working on a first-pass automation of a login flow for the Twitter Android app as part of [Automation Happy Hour Episode 6](https://ui.headspin.io/university/feed/automation-happy-hour-6), I ran into an interesting problem, which is illustrated by the screenshot below.\n\n

\n \"Twitter\n

\n\nTo get from the splash page to the login page, I needed to have Appium tap this \"Log in\" button. The only issue was, the button and the preceding text were all part of a single element. I could find the element, but when I used `element.click()` on it, nothing happened.\n\nAt first, I thought this was one of those cases where the element is not appropriately responding to the behavior of the `click` command. In such cases, it's often useful to make use of something I call a \"tap by location\" helper. This is a little helper method that makes it easy to tap an element using the [Actions API](https://appiumpro.com/editions/29-automating-complex-gestures-with-the-w3c-actions-api), instead of finding an element reference and calling `element.click()` as you normally would. Here's the basic approach:\n\n1. Find the element object\n2. Determine the position of the top-left corner of the element on the screen\n3. Determine the width and height of the element\n4. Use the result of #2 and #3 to find the midpoint of the element in terms of screen-based X and Y coordinates\n5. Use the Actions API to generate a tap at those coordinates\n\nIn code, it would look something like this, added to your base page object model:\n\n```java\nprotected void tapAtPoint(Point point) {\n AppiumDriver d = getDriver(); // assuming here a getDriver method\n PointerInput input = new PointerInput(Kind.TOUCH, \"finger1\");\n Sequence tap = new Sequence(input, 0);\n tap.addAction(input.createPointerMove(Duration.ZERO, Origin.viewport(), point.x, point.y));\n tap.addAction(input.createPointerDown(MouseButton.LEFT.asArg()));\n tap.addAction(new Pause(input, Duration.ofMillis(200)));\n tap.addAction(input.createPointerUp(MouseButton.LEFT.asArg()));\n d.perform(ImmutableList.of(tap));\n}\n\nprotected void tapElement(WebElement el) {\n Rectangle elRect = el.getRect();\n Point point = new Point(\n elRect.x + (int)(elRect.getWidth() / 2.0),\n elRect.y + (int)(elRect.getHeight() / 2.0)\n );\n tapAtPoint(point);\n}\n```\n\nBasically, we create a generic `tapAtPoint` method which handles turning a Point object into an Actions tap at that point. Then, we create the actual helper, `tapElement`, which does all the math for finding the midpoint of an element. To accomplish this it makes use of the `getRect` method for `WebElement`s, since that method returns to us both the X and Y location of the top-left corner of the element, and the dimensions. To find the midpoint, we just divide the width and height by 2, and add those quantities to the X and Y values for the top-left corner, respectively.\n\nNow, in our test code, we can use the helper as follows (using an element from TheApp as an example):\n\n```java\nWebElement el = driver.findElement(MobileBy.AccessibilityId(\"Login Screen\"));\ntapElement(el);\n```\n\nSo I used this approach with the Twitter app, and things *still* didn't work. What could be going on? Ultimately, I realized that it wasn't enough to just tap the element containing the 'Log in' text---I needed to tap that text specifically, even though it was all part of the same element as far as Appium was concerned. To make this possible, we would need to update our `tapElement` method, or more accurately, create a new helper method called `tapElementAt`. We need the ability to tap an element at an arbitrary distance from the top-left corner, and not just at the midpoint. So here's the set of helper methods we end up with when we make that addition and refactor:\n\n```java\nprotected void tapAtPoint(Point point) {\n AppiumDriver d = getDriver();\n PointerInput input = new PointerInput(Kind.TOUCH, \"finger1\");\n Sequence tap = new Sequence(input, 0);\n tap.addAction(input.createPointerMove(Duration.ZERO, Origin.viewport(), point.x, point.y));\n tap.addAction(input.createPointerDown(MouseButton.LEFT.asArg()));\n tap.addAction(new Pause(input, Duration.ofMillis(200)));\n tap.addAction(input.createPointerUp(MouseButton.LEFT.asArg()));\n d.perform(ImmutableList.of(tap));\n}\n\nprotected void tapElement(WebElement el) {\n tapElementAt(el, 0.5, 0.5);\n}\n\nprotected void tapElementAt(WebElement el, double xPct, double yPct) {\n Rectangle elRect = el.getRect();\n Point point = new Point(\n elRect.x + (int)(elRect.getWidth() * xPct),\n elRect.y + (int)(elRect.getHeight() * yPct)\n );\n tapAtPoint(point);\n}\n```\n\nNow, `tapElement` becomes just a special case of `tapElementAt`, with hard-coded values for the element-relative locations to tap. Notice that we are using relative percent values to pass into `tapElementAt`, rather than pixels. This makes it easy for us to eyeball the element regardless of screen size, and enter values that make sense. For the Twitter app above, I ended up writing the equivalent of:\n\n```\ntapElementAt(el, 0.9, 0.5);\n```\n\nThis generated a tap 90% of the way from the left edge of the element (so just 10% in from the right edge), and 50% down from the top (so right in the vertical middle). And this did the trick! I recommend adding this kind of element tap helper to your framework for situations you encounter like this. And if you want to see these methods in action, head on over to the [example source code](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition119_Tap_Helper.java) for this edition.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Using a Tap-By-Location Helper for Working With Unresponsive Elements","shortDesc":"Sometimes, finding an element isn't enough to tap on it. You call the element.click() command, but nothing happens! By building a tap-by-location helper, you might be able to get things working agian, especially if the problem is that you need to tap somewhere other than the midpoint of your element.","liveAt":"2020-07-08 00:00","canonicalRef":"https://www.headspin.io/blog/using-a-tap-by-location-helper-for-working-with-unresponsive-elements"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/119-using-a-tap-by-location-helper-for-working-with-unresponsive-elements","slug":"119-using-a-tap-by-location-helper-for-working-with-unresponsive-elements","path":"/editions/119-using-a-tap-by-location-helper-for-working-with-unresponsive-elements","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0118.md","num":118,"rawMd":"\n\n

\n \"Two\n

\n\nMany of the apps that we use, and need to test, involve the real-time interaction of multiple users, who are sitting in front of their own devices or browsers. The \"standard\" Appium test involves one driver object, representing one Appium session, automating a single device. How can we extend our usage of Appium so that we can *coordinate* multiple devices, the way multiple users would spontaneously coordinate their own behavior?\n\n### Examples of multi-user interaction\n\nFirst, let's take a look at some common use cases for a multi-user or multi-device test:\n\n1. A chat application involving one or more users interacting in real time\n2. A ridesharing app, involving two parties---the rider, who requests a ride using an app, and the driver, who can accept and pick up a rider using the same app. Rider and driver stay in constant interaction through the updating of a map that shows their positions.\n3. A food delivery app, involving three parties---the person requesting the food, the restaurant preparing the food, and the delivery person who picks up the food from the restaurant and delivers it to the requester.\n\nOf course, lots of other apps involve user interaction that isn't necessarily real-time. Any social app, for example, will have the ability for users to like or comment on other users' posts. In a normal case, it would be possible to conceive of this as happening asynchronously, not in real time. But many of these same apps also want to be able to test that the UI is updated in real time, when another user interacts with some content. So, for example, when I'm using Twitter, I get an in-app notification (rather than a system notification), if someone likes or retweets one of my posts. Testing this particular feature requires the same kind of multi-user interaction we're talking about here.\n\n### Background for a multi-device flow\n\nThe basic point I want to convey in this guide is that *Appium needs no special help to accommodate a multi-device, multi-user, real-time test scenario*. This is true because of the following facts about Appium (which you might already be aware of):\n\n1. Appium sessions are represented by objects in your test code. There's nothing to stop you from having multiple such objects.\n2. Appium servers can handle multiple simultaneous sessions (with appropriate config to make sure sessions don't step on each other's toes).\n\nThese two facts make it technically trivial (if not always intuitive from a test development perspective) to implement real-time multi-device flows. Now, there is a little caveat on number 2 above---because Appium sessions utilize certain system resources (primarily TCP ports) to communicate with specific devices, if you want to run multiple Appium sessions at once on a single machine, you'll need to include some capabilities that direct Appium to use specific ports for each session. This info has already been covered in the Appium Pro article on [running multiple Appium tests in parallel](https://appiumpro.com/editions/28-running-multiple-appium-tests-in-parallel). There is no difference from Appium's perspective between the same test being run in parallel on different devices, or one test incorporating multiple devices to test a user interaction flow.\n\n### Our multi-user AUT\n\nTo showcase multi-device testing with Appium, I developed a simple chat application, called [Appium Pro Chat](https://chat.appiumpro.com). It's a non-persisent multi-channel chat room, with zero memory, data storage, or (I hesitate to add) security. Feel free to go try it out in your browser! You can open up two browsers or tabs, pick a username, and join the same channel to see it in action. The basic test that I will implement is this:\n\n1. User 1 pick a username and join a channel\n2. User 2 pick a username and join the same channel as User 1\n3. User 1 and User 2 both type things into the chat\n4. Assert that the final chat log contains both of their messages in the correct order\n\nIt's a simple test, but one that has to involve the coordination of multiple Appium devices and multiple Appium sessions.\n\n### Multi-device setup\n\nFor this example, I am going to use an iOS simulator for User 1, and an Android device (Galaxy S7) for User 2. The first thing I need to do is make sure that I can start an Appium session on each of these, so I've created a helper method for each that just returns an Appropriate driver instance.\n\n```java\nprivate IOSDriver getSafari() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"13.3\");\n capabilities.setCapability(\"deviceName\", \"iPhone 11\");\n capabilities.setCapability(\"browserName\", \"Safari\");\n capabilities.setCapability(\"automationName\", \"XCUITest\");\n return new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n}\n\nprivate AndroidDriver getChrome() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"browserName\", \"Chrome\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n return new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n}\n```\n\nNow, in my test `setUp` method, I make sure to instantiate *both* drivers, which are stored as fields in my class, for example (omitting the other features of this class):\n\n```java\npublic class Edition118_Multi_Device {\n private IOSDriver safari;\n private AndroidDriver chrome;\n\n @Before\n public void setUp() throws MalformedURLException {\n safari = getSafari();\n chrome = getChrome();\n }\n\n @After\n public void tearDown() {\n if (safari != null) {\n safari.quit();\n }\n if (chrome != null) {\n chrome.quit();\n }\n }\n}\n```\n\nYou can see that there's nothing special to get multiple sessions going at the same time---I just first create the session on Safari on iOS, then the one on Chrome on Android. Setup and teardown now handle two drivers, instead of one. In this example, I am instantiating these sessions serially; if I wanted to optimize further, I could potentially instantiate them at the same time in separate threads, but it adds unnecessary complexity at this point.\n\n### Coordinating multiple drivers within a single test\n\nNow, we need to use both of these drivers (the `safari` and `chrome` objects) in such a way that they interact in real time with one another, in the course of a single test. Before I show the test method itself, I'll explain the plan: each of the two users will trade lines from the powerful Langston Hughes poem [Harlem](https://www.poetryfoundation.org/poems/46548/harlem). To encapsulate the data which will be used to support the test, we'll add it all as a set of fields at the top of the class:\n\n```java\nprivate static final String CHAT_URL = \"https://chat.appiumpro.com\";\nprivate static final String CHANNEL = \"Harlem\";\nprivate static final String USER1_NAME = \"Langston\";\nprivate static final String USER2_NAME = \"Hughes\";\nprivate static final ImmutableList USER1_CHATS = ImmutableList.of(\n \"What happens to a dream deferred?\",\n \"Or fester like a sore---and then run?\",\n \"Or crust and sugar over---like a syrupy sweet?\",\n \"Or does it explode?\");\nprivate static final ImmutableList USER2_CHATS = ImmutableList.of(\n \"Does it dry up like a raisin in the sun?\",\n \"Does it stink like rotten meat?\",\n \"Maybe it just sags like a heavy load.\",\n \"........yes\");\n```\n\n(Note that I have, perhaps ill-advisedly, but I believe in line with the spirit of the poem, added a final line of my own, partly to make User 2's list of messages equal the length of User 1's). With the data set up in `ImmutableList`s, we're in a position to iterate over the chats and implement the user interaction in a very concise way.\n\nNext, I wanted to create a set of helper methods to make it possible to reuse functionality between the two users. Both users will have to login and join a channel, and both users will have to send messages. Likewise, we can read the chat log from the perspective of either user. So we can factor all of this out into appropriate helper methods:\n\n```java\nprivate void joinChannel(RemoteWebDriver driver, String username, String channel) throws MalformedURLException {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n driver.navigate().to(new URL(CHAT_URL));\n wait.until(ExpectedConditions.presenceOfElementLocated(By.cssSelector(\"#channel\"))).sendKeys(channel);\n driver.findElement(By.cssSelector(\"#username\")).sendKeys(username);\n driver.findElement(By.cssSelector(\"#joinChannel\")).click();\n}\n\nprivate void sendChat(RemoteWebDriver driver, String message) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n wait.until(ExpectedConditions.presenceOfElementLocated(By.cssSelector(\"#sendMessageInput\"))).sendKeys(message);\n driver.findElement(By.cssSelector(\"#sendMessageBtn\")).click();\n}\n\nprivate String getChatLog(RemoteWebDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n return wait.until(ExpectedConditions.presenceOfElementLocated(By.cssSelector(\"#messages\"))).getText();\n}\n```\n\nNote that each of these takes a `RemoteWebDriver` object, because that is the appropriate superclass of both `IOSDriver` and `AndroidDriver`. We don't know within these methods whether we're automating iOS Safari or Android Chrome, and that's a good thing. It keeps the code general.\n\nNow, we can code up the test itself:\n\n```java\n@Test\npublic void testChatApp() throws MalformedURLException {\n joinChannel(safari, USER1_NAME, CHANNEL);\n joinChannel(chrome, USER2_NAME, CHANNEL);\n for (int i = 0; i < USER1_CHATS.size(); i++) {\n sendChat(safari, USER1_CHATS.get(i));\n sendChat(chrome, USER2_CHATS.get(i));\n }\n System.out.println(getChatLog(chrome));\n try { Thread.sleep(4000); } catch (Exception ign) {}\n}\n```\n\nAs you can see, it's fairly compact. We cause each of the clients to join the same channel, then iterate over their respective list of chat messages, sending each one to the channel. Finally, I retrieve the chat log from one of the users, and print it out to the console. Of course, in an actual test, we could make assertions on the content of the messages box, or even assert appropriate similarity between the chat logs seen by each user. Finally, for visual impact while running the test, we wait for 4 seconds before shutting everything down (again, this would not be the case in an actual test).\n\n### Recap\n\nThat's really all there is to it! Let me highlight the strategy we've taken here, in a way that could be adapted to any multi-user flow:\n\n1. Define the number of users required for the test.\n2. In setup, instantiate one Appium driver object for each participating user, taking care to follow all the rules for running Appium sessions in parallel.\n3. In the test itself, interleave commands for the separate drivers, following the logic of an actual user flow (usually these happen serially, since behavior for one user is dependent on behavior for another user, but if possible and appropriate, commands could run simultaneously in different threads).\n4. In teardown, make sure to clean up all the drivers.\n\nWant to have a look at the full code sample for this multi-user flow? [Check it out on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition118_Multi_Device.java)! And by all means, let me know if you've implemented any interesting multi-user or multi-device flows with Appium.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Testing Real-Time User Interaction Using Multiple Simultaneous Appium Sessions","shortDesc":"Many apps are not 'single-player,' so to speak. They involve the interaction of multiple users, sometimes in real time. Classic examples would be ride hailing, food delivery, or chat apps. While it's possible to mock out all except one of the parties, to get full end-to-end coverage of these scenarios, it can be useful to run multiple simultaneous Appium sessions, on different devices, to act out the various flows for the various participants, all just as it would happen in an actual use case.","liveAt":"2020-06-04 00:00","canonicalRef":"https://www.headspin.io/blog/testing-real-time-user-interaction-using-multiple-simultaneous-appium-sessions"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/118-testing-real-time-user-interaction-using-multiple-simultaneous-appium-sessions","slug":"118-testing-real-time-user-interaction-using-multiple-simultaneous-appium-sessions","path":"/editions/118-testing-real-time-user-interaction-using-multiple-simultaneous-appium-sessions","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0117.md","num":117,"rawMd":"\n\n

\n \"Appium\n

\n\nOK everyone, I have a candid confession to make. I started writing this article as a way to link up \"Appium\" and \"RPA (Robotic Process Automation)\", since RPA is such a hot thing right now. I even found an appropriately vague and generic technology graphic to accompany it. (Did you know that [this graphic](https://pixabay.com/photos/turn-on-turn-off-industry-energy-2923046/) came with the following tags: Industry, Energy, Power, and Businessman? I kid you not). But the more I tried to write some SEO gold, the more I realized how ridiculous this all is. So you're just going to get my honest thoughts here.\n\nRobotic Process Automation (or RPA) has become a huge industry. [This article](https://www.cio.com/article/3236451/what-is-rpa-robotic-process-automation-explained.html) has a wordy but standard definition for RPA:\n\n> RPA is an application of technology, governed by business logic and structured inputs, aimed at automating business processes. Using RPA tools, a company can configure software, or a “robot,” to capture and interpret applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems.\n\nOnce you parse out all the nonsense from this definition, what you arrive at is the idea of ... software controlling software! At a first pass, this is kind of a ridiculous idea. Software is already programmed to do things. Why do we need a second level of software to control other software? Why not just make that basic software do what we want to begin with? This is, in fact, what we should do if we can. Adding layers of indirection is never a good idea. Sometimes, though, we wind up in a situation where we can't actually modify the software we want to control. (Does this sound familiar to any of testers out there?) In that case, we can write additional software to *use* the first piece of software, *as if it were a user*.\n\nBefore everyone just says, \"wait a minute, that's exactly what we do with UI test automation!\" let me make one more point. The main motivation for RPA in my opinion is the proliferation of software as the primary mechanism of business. Employees are faced with the same kinds of mechanical, menial tasks they were faced with before the software revolution, but now they just happen on computer screens instead of in assembly lines. In this context, it makes sense to ask how software that you don't own, but need to use in various robotic ways, can be controlled. And the answer, as always, is \"robots!\" When it comes to manufacturing cars, that means physical robots. When it comes to entering data into spreadsheets, that means software robots.\n\nSo now, let's say it together: RPA is no different from UI test automation. It has a different *purpose*, sure, but the nuts and bolts are identical. All the tools you use for UI automation can also be used for RPA! So let's look at how Appium fits into all this. What makes Appium a good RPA tool?\n\n1. Appium supports multiple platforms. Want to automate processes on a Windows app? A macOS app? A mobile app? Together with Selenium, we've got you covered for any platform your software runs on.\n2. Appium gives you full UI control. RPA is typically black-box automation, just like UI testing, and that is where Appium focuses.\n3. Appium allows you to automate apps you don't own. This is essential for RPA, because if you owned the software you want to automate, you could just hook into its subroutines directly. That is usually not the case.\n4. Appium is a standalone server with clients written in any language, so you can get your RPA going using the language and tooling you already know.\n5. Appium, from a tooling perspective, is just a library you import, so it imposes no other restrictions on your workflow.\n\nOf course, using Appium as a scripting solution is likely not going to appeal to the audience that wants a fully point-and-click RPA development environment. However, with tools like [Appium Desktop](https://github.com/appium/appium-desktop), the barrier is slightly reduced, and there are already products built on top of Appium that allow for a fully visual experience.\n\nSo, next time you stumble across an article on RPA like the one above, and see 10-step guidelines that include phrases like \"5. Don't fall down the data rabbit hole\" or \"10. Put RPA into your whole development lifecycle\", you safely discard it as unnecessary fluff. All you really need to know about RPA is that it is UI automation, plain and simple. The rest is details. I'm not implying that there is no business or managerial complexity to implementing RPA solutions. I'm merely pointing out to all the CIOs out there that their \"RPA Center of Excellence\" is already staring them in the face in the form of their test automation team!\n\nOK, now you can go and add \"RPA\" to your resume and ask for a raise. You're welcome!\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Using Appium for Robotic Process Automation (RPA)","shortDesc":"RPA is a massive and growing part of the software industry, focused around the automation of repetitive business processes. While it has its own name and its own use cases, RPA is really not distinct from the tools and processes used in UI test automation. In this edition we take a look at how you can use Appium for all your RPA needs.","liveAt":"2020-05-27 00:00","canonicalRef":"https://www.headspin.io/blog/using-appium-for-robotic-process-automation-rpa"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/117-using-appium-for-robotic-process-automation-rpa","slug":"117-using-appium-for-robotic-process-automation-rpa","path":"/editions/117-using-appium-for-robotic-process-automation-rpa","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0116.md","num":116,"rawMd":"\n\nSometimes you want to automate an iOS device, but don't want to automate any app in particular, or want to start from the homescreen as part of a multi-app flow, or simply want to automate a set of built-in apps the way a user would approach things. In this case, it's actually possible to start an Appium iOS session *without* a specific app.\n\nTo do this we make use of the concept of the iOS [Springboard](https://en.wikipedia.org/wiki/SpringBoard), which is essentially another word for the home screen. The Springboard is essentially an app, though it's one that can't be terminated. As an \"app\", it has its own bundle ID: `com.apple.springboard`! So we can actually use this to start an Appium session without referring to any \"real\" app in particular:\n\n```java\ncapabilities.setCapability(\"app\", \"com.apple.springboard\");\n```\n\nOn its own, however, this isn't going to work, because Appium will try to launch this app, and deep in the XCUITest code related to app launching is some logic that makes sure the app is terminated before launch. As I mentioned earlier, the Springboard can't be terminated, so trying to start an Appium session this way will lead to a hanging server. What we can do is include another capability, `autoLaunch`, and set it to `false`, which tells Appium not to bother with initializing and launching the app, but just to start a session and give you back control immediately:\n\n```java\ncapabilities.setCapability(\"autoLaunch\", false);\n```\n\nAt this point, starting an Appium session in this way will drop you at the Springboard. It won't necessarily drop you at any particular *page* of the Springboard, however. If you're an iOS user, you'll know that the \"home screen\" is really a whole set of screens, depending on how many apps you have and how you've organized them. One of the main things you would want to do from the home screen is find and interact with an icon for a given app. So how can we do this?\n\n### Finding App Icons\n\nUltimately, what would be nice to have code-wise is a handy [custom expected condition](https://appiumpro.com/editions/114-how-and-why-to-create-a-custom-expected-condition). Let's imagine that this is our test method implementation, for example:\n\n```java\n@Test\npublic void testSpringboard() {\n wait.until(AppIconPresent(\"Reminders\")).click();\n pressHome();\n wait.until(AppIconPresent(\"Contacts\")).click();\n pressHome();\n}\n```\n\nHere we have created a custom expected condition called `AppIconPresent`, which takes the app icon text, and will attempt to find that icon, navigating through the different \"pages\" of the Springboard if the icon is not already present. This is actually conceptually a bit tricky, because of how the Springboard app is implemented. No matter how many pages you have in your Springboard, all pages show up within the current UI hierarchy. This means it is easy to find an icon for an app even if it's not on the currently-displayed page. However, if you try to *tap* that icon, it will not work (because the icon is not actually visible). So, we need some way of figuring out how to move to the correct page before tapping. Let's examine the implementation of `AppIconPresent`:\n\n```java\nprotected ExpectedCondition AppIconPresent(final String appName) {\n pressHome();\n curPage = 1;\n return new ExpectedCondition() {\n @Override\n public WebElement apply(WebDriver driver) {\n try {\n return driver.findElement(By.xpath(\n \"//*[@name='Home screen icons']\" +\n \"//XCUIElementTypeIcon[\" + curPage + \"]\" +\n \"/XCUIElementTypeIcon[@name='\" + appName + \"']\"\n ));\n } catch (NoSuchElementException err) {\n swipeToNextScreen();\n curPage += 1;\n throw err;\n }\n }\n };\n}\n```\n\nThe first thing we do is call our `pressHome` helper method (which is just another method implemented in the current class). It looks like this:\n\n```java\nprotected void pressHome() {\n driver.executeScript(\"mobile: pressButton\", ImmutableMap.of(\"name\", \"home\"));\n}\n```\n\n(I just have it as a nice helper to avoid redefining the executeScript args all the time). What calling `pressHome` here does is ensure that we are always on the first page of the Springboard. Then, we set a class field to define what page we are on. We initialize it to `1`, because after pressing the home button, we know we are on the first page. Then, in our actual condition check implementation, we try to find an icon that has the name we've been given to find.\n\nHere's the tricky part: we don't want to just find *any* icon that has the correct name (because then we would find the icon even if it's not on the current page). We only want to find an icon on the *current page* (and then swipe to the next page if we can't find it). To do that, we take advantage of a fact about Springboard's UI hierarchy, which is that each page is actually coded up as an `XCUIElementTypeIcon`, which contains the actual app icons as children. So given that we know what page we're on, we can write an XPath query that restricts our search to the XML nodes corresponding to the current page!\n\nIf we're unable to find an icon on the current page, we call another helper method, `swipeToNextScreen`. This is built off of the handy [swipe helper](https://appiumpro.com/editions/107-designing-a-cross-platform-swipescroll-helper) developed in an earlier post, and has a simple implementation that just performs a swipe from an area near the right edge of the screen over to the left:\n\n```java\nprotected void swipeToNextScreen() {\n swipe(0.9, 0.5, 0.1, 0.5, Duration.ofMillis(750));\n}\n```\n\nOnce we've swiped to the next screen, we increment our page counter because we have now moved to the next screen! Of course, we're relying on the assumption that we will eventually find the app by the time we reach the last page, because we don't have any logic to detect whether our `swipeToNextScreen` was actually successful.\n\nIn general, `AppIconPresent` is a great example of a useful custom expected condition that has a side effect. We build it into an expected condition so we can use it flexibly with the `WebDriverWait` interface, and so we don't need to write any of the looping or retry logic ourselves. It's also a great example of being clever (and hopefully not *too* clever) with XPath!\n\nSo that's it for automating the iOS home screen or \"Springboard\" app. Let me know if you find any other interesting uses for automating this app beyond using it as a launcher or a quick way to get an Appium session on a device! And as always, you can find the [full sample code](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition116_iOS_Springboard.java) for this edition up on GitHub.\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Working with the iOS Home Screen (Springboard)","shortDesc":"Appium usually requires an 'app' capability so that it can launch an app that you want to test. But what if you don't want to test any app in particular? In this case, we can decide to start with the 'Springboard' app, which is another name for the iOS home screen or app launcher.","liveAt":"2020-05-20 00:00","canonicalRef":"https://www.headspin.io/blog/working-with-the-ios-home-screen-springboard"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/116-working-with-the-ios-home-screen-springboard","slug":"116-working-with-the-ios-home-screen-springboard","path":"/editions/116-working-with-the-ios-home-screen-springboard","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0114.md","num":114,"rawMd":"\n\n

\n \"Automating\n

\n\nOne of the issues I ran into in the [Automation Happy Hour featuring Zoom (part 2)](https://ui.headspin.io/university/feed/automation-happy-hour-2), was that the Zoom UI disappears during a meeting, making it hard to tap on buttons in the UI. The way that a user would get around this is of course to (petulantly) tap the middle of the screen, which triggers the app to re-show the UI.\n\nGiven that we don't know which state the UI will be in when we want to interact with it, we're in a bit of a pickle with our automation. If we try to use a simple WebDriverWait for a UI element, it will fail, because the UI will never show up unless we take an action. However, if we blindly take an action without checking the state first, then we run the risk of actually *hiding* the UI instead of showing it (because maybe it was already present!).\n\nOne solution to this problem would be to create a helper method that wraps the logic for checking if the UI is present, and performs the appropriate tap if not:\n\n```java\nvoid ensureUIPresent() {\n try {\n driver.findElement(UI_ELEMENT);\n } catch (NoSuchElementException ign) {\n driver.findElement(SCREEN).click();\n }\n}\n```\n\nIn this simplistic logic, we first check if the appropriate UI element is present. In the Zoom app, this could be the \"Leave\" button, for example, which is always visible whenever the UI is present. If we can't find this element, then we find some large-scale screen element, and tap it (assuming we can find an element whose center is precisely in the middle of the screen). Then, we use this helper anywhere we want to be sure the UI is present! Pretty simple. It might be a little too simple, in fact--what if the UI is in the middle of transitioning? You could image we run into a situation where we tap the \"SCREEN\" element to show the UI, and pass control back to our caller, who then tries to use a UI element while the UI is still busy coming out of hiding.\n\nAnd there is a more elegant way to wrap up this solution in general, which is to create something called an `ExpectedCondition`. You might be familiar with `ExpectedCondition`s from using `WebDriverWait`. The Appium / Selenium Java client has a bunch of built in `ExpectedCondition`s that you can use, the most common of which is `presenceOfElementLocated`:\n\n```java\nwait.until(ExpectedConditions.presenceOfElementLocated(LOCATOR));\n```\n\nWe can also make our *own* `ExpectedCondition`s! All we have to do is return a class that overrides the appropriate method. Let's take our previous `ensureUIPresent` method and turn it into one of these `ExpectedCondition`s.\n\n```java\nprivate ExpectedCondition ZoomUIPresent() {\n return new ExpectedCondition() {\n @Override\n public Boolean apply(WebDriver driver) {\n try {\n driver.findElement(LEAVE_BTN);\n return true;\n } catch (NoSuchElementException ign) {\n driver.findElement(CONTENT).click();\n }\n return false;\n }\n };\n}\n```\n\nHere I've created an `ExpectedCondition` wrapper called `ZoomUIPresent`. In the returned condition, we override the `apply` method to return a `true` or `false` value. The WebDriver wait object will know how to use these return values to keep trying to wait for the expected condition to return true! The logic here is pretty straightforward:\n\n1. Try and find the 'Leave' button (which, if found, proves the UI is present)\n2. If we can find it, return true because we have satisfied our condition\n3. If we can't find it, click the 'content' element (which should trigger a tap to the center of the screen), and return false\n4. If we return false, the condition will run again until we return true!\n\n(Note that I have used `findElement` for simplicity, but we should probably be using a short WebDriverWait instead, to handle the potential race condition I described above).\n\nNow, to *use* this new expected condition, we just toss it into `WebDriverWait.until()`!\n\n```java\n// do some stuff\nwait.until(ZoomUIPresent());\n// do some other stuff now that we know the UI is present\n```\n\nSo this is a real world example of how and why you might want to create your own app-specific expected condition! Have you created your own expected conditions in the past? Let me know how you used them to make your own automation framework more functional and elegant!\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"How (And Why) To Create a Custom Expected Condition","shortDesc":"When waiting for app state, you're not limited to the built-in expected conditions from the Appium client libraries. You can also create your own expected conditions, which neatly wrap up information about app-specific state, and potentially include helpful side effects as well. As a case study, we examine a particular custom expected condition I created for use with automation of the Zoom app, following an Automation Happy Hour.","liveAt":"2020-04-29 08:00","canonicalRef":"https://www.headspin.io/blog/how-and-why-to-create-a-custom-expected-condition"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/114-how-and-why-to-create-a-custom-expected-condition","slug":"114-how-and-why-to-create-a-custom-expected-condition","path":"/editions/114-how-and-why-to-create-a-custom-expected-condition","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0113.md","num":113,"rawMd":"\n\n

\n \"Automating\n

\n\nLast week I tried something new: [Automation Happy Hour](https://ui.headspin.io/university/feed/automation-happy-hour-1). In this live stream event, I downloaded the Zoom app for Android and tried to build a simple testsuite for it in about an hour. I think this type of real-world \"case study\" is really interesting. My expectation is that discussing automation of real apps \"in the wild\" would also be useful for Appium Pro readers, so periodically, I will take the results of these explorations and turn them into Appium Pro articles!\n\n(NB: In the live stream, I chose to work in Python, but for this article I converted all the code examples to Java.)\n\n### Starting a session with Zoom\n\n[Zoom](https://zoom.us) is incredibly popular. It seems like everyone is using it for their video meetings these days. How can we launch it in an automation context with Appium?\n\nFirst, we need to decide whether to have Appium install the Zoom app for us, or to have the app on the phone and simply launch it when our session starts. I already had the Zoom app on my device, so I chose the latter route, but you could also find a downloadable Zoom APK, put it somewhere on your system, and reference it as the `app` capability.\n\nSince I had the app already, I needed to know its package id and launching activity to get Appium to start it for me. Using the helpful ADB command `adb dumpsys window windows`, I determined Zoom's info, and entered that into my set of capabilities:\n\n```java\ncaps.setCapability(\"platformName\", \"Android\");\ncaps.setCapability(\"deviceName\", \"Android\");\ncaps.setCapability(\"appPackage\", \"us.zoom.videomeetings\");\ncaps.setCapability(\"appActivity\", \"com.zipow.videobox.LauncherActivity\");\ncaps.setCapability(\"automationName\", \"UiAutomator2\");\n```\n\n(The important bits above are the `appPackage` and `appActivity` capabilities). Using these caps, we can launch zoom in an automation context!\n\n### Joining a meeting\n\nThe goal is to do something useful with automation, so let's see how it's possible to join a Zoom meeting from a device controlled by Appium. Before we begin, we'll obviously need a Zoom meeting to join! I started one on my computer, so that I could determine its meeting ID and password. These are the two pieces of information you'll need that are unique to your case, and to work with my code examples, you'll need to set them as environment variables, so that they can be read from our test class:\n\n```java\nprivate static final String MEETING_ID = System.getenv(\"ZOOM_MEETING_ID\");\nprivate static final String MEETING_PW = System.getenv(\"ZOOM_MEETING_PW\");\n```\n\n(I.e., you need to set the `ZOOM_MEETING_ID` and `ZOOM_MEETING_PW` environment vars to run the examples successfully, based on a meeting that you own or know about--and, please, no zoom-bombing!)\n\nIt turns out that Zoom is a highly automatable app! Its developers included resource IDs for all the useful elements. There were no content descriptions (accessibility IDs) that I could find, however, which makes me think the app may not be super accessible (or, it relies on the default accessibility strategies based on text labels).\n\nTo figure out how to find all the elements I needed to interact with, I used [Appium Desktop](https://github.com/appium/appium-desktop) to inspect the app (after filling out all the appropriate caps, and starting a session with the inspector). It turns out that the Zoom elements follow a pretty standard naming scheme. Here's the part of our test class where we define the locators, for example:\n\n```java\nprivate By JOIN_MEETING_BUTTON = By.id(\"btnJoinConf\");\nprivate By MEETING_ID_FIELD = By.id(\"edtConfNumber\");\nprivate By ACTUALLY_JOIN_MEETING_BUTTON = By.id(\"btnJoin\");\nprivate By PASSWORD_FIELD = By.id(\"edtPassword\");\nprivate By PASSWORD_OK_BUTTON = By.id(\"button1\");\nprivate By LEAVE_BTN = By.id(\"btnLeave\");\n```\n\nYou can see that, apart from `PASSWORD_OK_BUTTON` (which is referring to a system alert element), every UI element has a camelCase ID with an abbreviation of the element type at front. Pretty respectable, Zoom!\n\nUsing these locators, we can very straightforwardly walk through the join meeting flow, as the video below shows:\n\n

\n \"Joining\n

\n\nThe test code that implements this is as follows (using the locators defined above):\n\n```java\n@Test\npublic void testJoinMeeting() {\n // navigate through the UI to join a meeting with correct meeting id and password\n waitFor(JOIN_MEETING_BUTTON).click();\n waitFor(MEETING_ID_FIELD).sendKeys(MEETING_ID);\n driver.findElement(ACTUALLY_JOIN_MEETING_BUTTON).click();\n waitFor(PASSWORD_FIELD).sendKeys(MEETING_PW);\n driver.findElement(PASSWORD_OK_BUTTON).click();\n\n // prove that we made it into the meeting by finding the 'leave' button\n waitFor(LEAVE_BTN);\n}\n```\n\nHere we are using a special `waitFor` helper which just reduces verbosity in the code:\n\n```java\nprivate WebElement waitFor(By locator) {\n return wait.until(ExpectedConditions.presenceOfElementLocated(locator));\n}\n```\n\n(You might have observed that sometimes we use `waitFor` and sometimes we just use `driver.findElement`. Why is that? It wouldn't hurt to use `waitFor` everywhere, but in some cases we know that an element will already be present, because we've already waited for another element on the same view. If that first element is *not* actually present immediately, then there's no reason to wait however many seconds for it to become present. The test is doomed and we might as well fail quickly!)\n\nSo that's it! You can as always check out the [full code example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition113_Automating_Zoom.java) as well. And if you're interested in checking out some of the Automation Happy Hour live streams (or the video recordings of them), don't forget to [follow me on Twitter](https://twitter.com/jlipps), where I announce these things as well as run polls on what apps I should automate in future episodes.\n\nOh, and don't forget to [let me know](https://appiumpro.com/contact) if you come up with any interesting (and non-nefarious) use cases for automating Zoom specifically!\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"How to Automate the Zoom Video Meetings App","shortDesc":"Taking the results of a recent Automation Happy Hour, we explore the Zoom app. How automatable is it? How can we reliably automate the process of joining a meeting? Turns out it's totally doable!","liveAt":"2020-04-23 08:00","newsletterTopNote":"

This Friday (tomorrow) at 2p PDT, #AutomationHappyHour strikes again!. Join me at my Twitch channel then, and don't worry if you can't make it. I'll record the video of the stream for later viewing.","canonicalRef":"https://www.headspin.io/blog/how-to-automate-the-zoom-video-meetings-app"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/113-how-to-automate-the-zoom-video-meetings-app","slug":"113-how-to-automate-the-zoom-video-meetings-app","path":"/editions/113-how-to-automate-the-zoom-video-meetings-app","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0112.md","num":112,"rawMd":"\n\nIf you do a lot of iOS testing using Appium, and have requirements especially for user flows that involve rotating the device to a different screen orientation, you might have noticed that screenshots retrieved by Appium in these cases might be incorrectly rotated. That is, the screenshot might not match the visible orientation of the device. This can be especially troublesome if you are using image-based testing, because Appium uses the coordinates from the screenshot to enable you to interact with found image regions!\n\nUnfortunately, some of the time the screenshot comes back rotated incorrectly, Appium is not at fault; in these cases, XCUITest is just doing its thing, and Appium doesn't know whether the image is correct or not (it's not peeking at the \"contents\" of the screenshot to check that it's correct). But if you find yourself in this situation as a test author, *you* know what the screenshot is supposed to look like, *you* know which orientation your device is in, and you can tell Appium which orientation you want it to use for screenshots!\n### The `screenshotOrientation` setting\n\nTo accomplish this we make use of Appium's Settings API, and specifically a setting (released in Appium 1.17) called `screenshotOrientation`. There are 5 possible values this setting can take:\n\n* `auto`: don't enforce any screenshot orientation. Just pass along the screenshot given by XCUITest.\n* `portrait`: enforce portrait orientation for the screenshot.\n* `portraitUpsideDown`: enforce the \"upside-down\" orientation for the screenshot.\n* `landscapeRight`: enforce a landscape (turned to the right) orientation.\n* `landscapeLeft`: enforce a landscape (turned to the left) orientation.\n\nThe way we tell Appium to use a particular value is (in Java) with the `driver.setSetting` command:\n\n```java\ndriver.setSetting(\"screenshotOrientation\", \"landscapeRight\")\n```\n\nCalling this will set the screenshot orientation value for the duration of the Appium session (or until I choose to call `setSetting` again with a different value).\n\n### Usage example\n\nI was able to see this setting in action by writing a test that first turns a device to landscape mode, and then attempts to take a screenshot. If, before I start the test, I rotated the device manually, but in the opposite direction to the one Appium would normally rotate it (i.e., if I rotate it left), then I noticed that Appium's screenshots came back upside down!\n\nAnyway, in this case, including the setting of the `screenshotOrientation` value enabled me to guarantee the screenshot would always come back in the correct orientation. Check out the full example below (and as always, the code is up [on GitHub too](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition112_iOS_Screenshot_Orientation.java):\n\n```java\npublic class Edition112_iOS_Screenshot_Orientation {\n private IOSDriver driver;\n private final static String PHOTOS = \"com.apple.mobileslideshow\";\n\n @Before\n public void setUp() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"13.3\");\n capabilities.setCapability(\"deviceName\", \"iPad Pro (12.9-inch) (3rd generation)\");\n capabilities.setCapability(\"app\", PHOTOS);\n capabilities.setCapability(\"simulatorTracePointer\", true);\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testScreenshots() throws Exception {\n Thread.sleep(2000);\n String desktop = System.getenv(\"HOME\") + \"/Desktop\";\n driver.rotate(ScreenOrientation.LANDSCAPE);\n\n // this screenshot might be correct or it might be upside down\n File regularScreenshot = driver.getScreenshotAs(OutputType.FILE);\n\n // this screenshot should always be correct\n driver.setSetting(\"screenshotOrientation\", \"landscapeRight\");\n File adjustedScreenshot = driver.getScreenshotAs(OutputType.FILE);\n FileUtils.copyFile(regularScreenshot, new File(desktop + \"/screen1.png\"));\n FileUtils.copyFile(adjustedScreenshot, new File(desktop + \"/screen2.png\"));\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"How to Specify Screenshot Orientation for iOS Devices","shortDesc":"Sometimes, for reasons that are currently unfixable, Appium returns screenshots on iOS that don't match the actual orientation of the device screen. This varies based on type of device, iOS version, etc... Luckily, Appium has a special setting you can use to always enforce a certain orientation for the screenshot, if you know what it should be.","liveAt":"2020-04-15 10:00","newsletterTopNote":"

Oh, and special news! This Friday 4/17 at 2p PDT, I'll be live-streaming some random app automation fun under the banner of #AutomationHappyHour. Join me then at my Twitch channel to join in. If we all have fun maybe we'll make it a regular thing. I'll see if it's possible to record for those who can't make it live, too.","canonicalRef":"https://www.headspin.io/blog/how-to-specify-screenshot-orientation-for-ios-devices"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/112-how-to-specify-screenshot-orientation-for-ios-devices","slug":"112-how-to-specify-screenshot-orientation-for-ios-devices","path":"/editions/112-how-to-specify-screenshot-orientation-for-ios-devices","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0111.md","num":111,"rawMd":"\n\nAs Covid-19 sweeps around the globe and brings a lot of unexpected change and complexity in people's lives, whether through their becoming ill or because of the social and economic consequences of this crisis, I've been asking myself: what can Appium do to help? Automating mobile apps pales in comparison to the work that healthcare professionals are doing right now, and certainly doesn't feel sufficient to address the severeness of the concrete need currently being felt in so many communities all around the world.\n\nI tried to think of a clever way to use mobile automation to trigger donations to appropriate services, [like we did once before](https://appiumpro.com/editions/49). When it really comes down to it, though, that's a gimmicky sort of response to a crisis like the one we're in. What people need most right now may not be mobile app automation. It's pretty cool that Jason Huggins can [repurpose his robot automation business to make face shields](https://twitter.com/hugs/status/1242221899884695553) for healthcare personnel, but that level of physical aid is probably beyond us on the software automation side.\n\nBut you know what? That's OK. It turns out (at least I hope it does) that we are human beings first, and Appium users second (or twentieth, or what have you). The real question is, what can we, as human beings, do for our fellow human beings in this crisis? And not just for our \"fellow human beings\" considered abstractly, but our actual physical neighbor humans that live next door and downstairs and across the street? Obviously the mandate to stay home and prevent the spread of disease through physical contact limits our options. It doesn't eliminate them, though.\n\nThis is a hugely unpredictable time in the world, and the economic fallout looks like it may be as severe as the health risks have already proven to be. Some of you reading this may have lost your jobs. For you, I hope that the work I've started to [bring high-quality test automation instruction online](https://cloudgrey.io/blog/jonathan-lipps-joins-headspin) will be of some use in helping to find good employment in an uncertain time.\n\nFor others, the sentiment may be one of more or less relief. Compared to many other industries, tech hasn't seemed poised to be altered so dramatically. Because of the Internet, we can all work from home. Because of the Internet, people can continue to consume the value that is generated through mobile app development. For those of us in this situation, we are in a uniquely fortunate position to help others. It might not be wise for us to go babysit our neighbor's kids, but we *could* send our neighbor money for groceries this week. It might not be wise for us to eat out at our local restaurant or coffee shop, but we *could* order food for the whole block and help keep them in business. It might not be wise to get a haircut or take a yoga class, but we *could* pay for those services anyway, sending a message that we value the service providers enough to help keep them afloat during a season that is uniquely difficult for their businesses.\n\nI sure hope that governments around the world are prioritizing taking care of their citizens in these times, including taking care of their physical and financial needs. But for those of us who might be quite fortunate relative to others right now, it's worth asking the question of how *we* can care for our communities in those ways too, especially when it's so easy, safe, and quick to send some cash online. I know that this is supposed to be a newsletter full of technical tips. Don't worry: we've got [over 100 of those](https://appiumpro.com/editions) for you to browse! And there will be many more to come. But today, I wanted to encourage each of us to take a step back from our computer screens, look around for some specific need in our community, and meet it. I just did this a little bit ago myself, and without going into detail, I'll just say that it was an immensely good thing.\n\nOh, and if you *do* think of a way to use Appium to help people in this crisis, or to fight the Coronavirus, let me know! I'll help you get the word out.\n\nAlright Appium humans, let's do this! Stay healthy, stay safe, and stay generous. Hopefully we'll all get back to our regularly scheduled app automation very soon, and maybe even with a little more empathy for everyone else we're \"sheltering in place\" with on this big chunk of rock floating in the starry void.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Appium in the Time of Coronavirus","shortDesc":"This is not a typical Appium Pro article. It's a call for kindness and compassion in the midst of the first, and hopefully the last, major global crisis during the tenure of Appium Pro.","liveAt":"2020-04-01 10:00","canonicalRef":"https://www.headspin.io/blog/appium-in-the-time-of-coronavirus"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/111-appium-in-the-time-of-coronavirus","slug":"111-appium-in-the-time-of-coronavirus","path":"/editions/111-appium-in-the-time-of-coronavirus","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0110.md","num":110,"rawMd":"\n\n

\n \"Appium\n

\n\nDid you know that Appium is just a web server? That's right! I'm going to write a full edition at some point on the WebDriver Protocol, which is the API that Appium implements (and which, as you can tell by the name, was invented for the purpose of browser automation). But for now, let's revel in the knowledge that Appium is just like your backend web or API server at your company. In fact, you could host Appium on a server connected to the public internet, and give anybody the chance to run sessions on your devices! (Don't do this, of course, unless you're a cloud Appium vendor).\n\nYou may be used to writing Appium code in Java or some other language, that looks like this:\n\n```java\ndriver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\nWebElement el = driver.findElement(locator);\nSystem.out.println(el.getText());\ndriver.quit();\n```\n\nThis looks like Java code, but what's happening under the hood is that each of these commands is actually triggering an HTTP request to the Appium server (that's why we need to specify the location of the Appium server on the network, in the `IOSDriver` constructor). We could have written all the same code, for example, in Python:\n\n```py\ndriver = webdriver.Remote(\"http://localhost:4723/wd/hub\", capabilities)\nel = driver.find_element(locator)\nprint(el.text)\ndriver.quit()\n```\n\nIn both of these cases, while the surface code looks different, the underlying HTTP requests sent to the Appium server (and the responses coming back) are the same! This is what allows Appium (and Selenium, where we stole this architecture from) to work in any programming language. All someone needs to do is code up a nice little client library for that language, that converts that language's constructs to HTTP requests.\n\nWhat all this means is that we technically don't need a client library at all. It's convenient to use one, absolutely. But sometimes, we want to just run an ad-hoc command against the Appium server, and creating a whole new code file and trying to remember all the appropriate syntax might be too much work. In this case, we can just use [curl](https://curl.haxx.se/), which is a command line tool used for constructing HTTP requests and showing HTTP responses. Curl works on any platform, so you can download it for your environment if it's not there already (it comes by default on Macs, for example). There are lots of options for using curl, and to use it successfully on your own, you should understand all the components of HTTP requests. But for now, let's take a look at how we might encode the previous four commands, without any Appium client at all, just by using curl!\n\n```bash\n# 0. Check the Appium server is online\n> curl http://localhost:4723/wd/hub/status\n\n# response:\n{\"value\":{\"build\":{\"version\":\"1.17.0\"}},\"sessionId\":null,\"status\":0}\n```\n\nYou can see above that we can make sure we have a connection to the Appium server just by running `curl` and then the URL we want to retrieve, in this case the `/status` endpoint. We don't need any parameters to curl other than the URL, because we're making a GET request, and so no other parameters are required. The output we get back is a JSON string representing Appium's build information. Now, let's actually start a session:\n\n```bash\n# 1. Create a new session\n> curl -H 'Content-type: application/json' \\\n -X POST \\\n http://localhost:4723/wd/hub/session \\\n -d '{\"capabilities\": {\"alwaysMatch\": {\"platformName\": \"iOS\", \"platformVersion\": \"13.3\", \"browserName\": \"Safari\", \"deviceName\": \"iPhone 11\"}}}'\n\n# response:\n{\"value\":{\"capabilities\":{\"webStorageEnabled\":false,\"locationContextEnabled\":false,\"browserName\":\"Safari\",\"platform\":\"MAC\",\"javascriptEnabled\":true,\"databaseEnabled\":false,\"takesScreenshot\":true,\"networkConnectionEnabled\":false,\"platformName\":\"iOS\",\"platformVersion\":\"13.3\",\"deviceName\":\"iPhone 11\",\"udid\":\"140472E9-8733-44FD-B8A1-CDCFF51BD071\"},\"sessionId\":\"ac3dbaf9-3b4e-43a2-9416-1a207cdf52da\"}}\n\n# save session id\n> export sid=\"ac3dbaf9-3b4e-43a2-9416-1a207cdf52da\"\n```\n\nLet's break this one down line by line:\n\n1. Here we invoke the `curl` command, passing the `-H` flag in order to set an HTTP request header. The header we set is the `Content-type` header, with value `application/json`. This is so the Appium server knows we are sending a JSON string as the body of the request. Why do we need to send a body? Because we have to tell Appium what we want to automate (our \"capabilities\")!\n2. `-X POST` tells curl we want to make a POST request. We're making a POST request because the [WebDriver spec](https://github.com/jlipps/simple-wd-spec#new-session) defines the new session creation command in a way which expects a POST request.\n3. We need to include our URL, which in this case is the base URL of the Appium server, plus `/session` because that is the route defined for creating a new session.\n4. Finally, we need to include our capabilities. This is achieved by specifying a POST body with the `-d` flag. Then, we wrap up our capabilities as a JSON object inside of an `alwaysMatch` and a `capabilities` key.\n\nRunning this command, I see my simulator pop up and a session launch with Safari. (Did the session go away before you have time to do anything else? Then make sure you set the `newCommandTimeout` capability to `0`). We also get a bunch of output like in the block above. This is the result of the new session command. The thing I care most about here is the `sessionId` value of `ac3dbaf9-3b4e-43a2-9416-1a207cdf52da`, because I will need this to make future requests! Remember that HTTP requests are stateless, so for us to keep sending automation commands to the correct device, we need to include the session ID for subsequent commands, so that Appium knows where to direct each command. To save it, I can just `export` it as the `$sid` shell variable.\n\nNow, let's find an element! There's just one element in Appium's little Safari welcome page, so we can find it by its tag name:\n\n```bash\n# 2. Find an element\n> curl -H 'Content-type: application/json' \\\n -X POST http://localhost:4723/wd/hub/session/$sid/element \\\n -d '{\"using\": \"tag name\", \"value\": \"h1\"}'\n\n# response:\n{\"value\":{\"element-6066-11e4-a52e-4f735466cecf\":\"5000\",\"ELEMENT\":\"5000\"}}\n\n# save element id:\n> export eid=\"5000\"\n```\n\nIn the curl command above, we're making another POST request, but this time to `/wd/hub/session/$sid/element`. Note the use of the `$sid` variable here, so that we can target the running session. This route is the one we need to hit in order to find an element. When finding an element with Appium, two parameters are required: a locator strategy (in our case, \"tag name\") and a selector (in our case, \"h1\"). The API is designed such that the locator strategy parameter is called `using` and the selector parameter is called `value`, so that is what we have to include in the JSON body.\n\nThe response we get back is itself a JSON object, whose value consists of two keys. The reason there are two keys here is a bit complicated, but what matters is that they each convey the same information, namely the ID of the element which was just found by our search (`5000`). Just like we did with the session ID, we can store the element ID for use in future commands. Speaking of future commands, let's get the text of this element!\n\n```bash\n# 3. Get text of an element\n> curl http://localhost:4723/wd/hub/session/$sid/element/$eid/text\n\n# response:\n{\"value\":\"Let's browse!\"}\n```\n\nThis curl command is quite a bit simpler, because retrieving the text of an element is a GET command to the endpoint `/session/$sid/element/$eid/text`, and we don't need any additional parameters. Notice how here we are using both the session ID and the element ID, so that Appium knows which session and which element we're referring to (because, again, we might have multiple sessions running, or multiple elements that we've found in a particular session). The response value is the text of the element, which is exactly what we were hoping to find! Now all that's left is to clean up our session:\n\n\n```bash\n# 4. Quit the session\n> curl -X DELETE http://localhost:4723/wd/hub/session/$sid\n\n# response:\n{\"value\":null}\n```\n\nThis last command can use all the default curl arguments except we need to specify that we are making a DELETE request, since that is what the WebDriver protocol requires for ending a session. We make this request to the endpoint `/session/$sid`, which includes our session ID so Appium knows which session to shut down.\n\nThat's it! I hope you've enjoyed learning how to achieve some \"low level\" HTTP-based control over your Appium (and Selenium) servers!\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Controlling Appium via raw HTTP requests with curl","shortDesc":"At the end of the day, Appium and Selenium are just web servers. That means we can interact with Appium just like we could with any other REST API, including using command-line tools like curl to generate raw HTTP requests directly to an Appium automation session.","liveAt":"2020-03-27 10:00","newsletterTopNote":"Sorry for the periodic delays on this weekly newsletter! A lot has been going on for me personally during the global pandemic--rest assured the Appium will keep flowing, if slightly more intermittently at the moment.","canonicalRef":"https://www.headspin.io/blog/controlling-appium-via-raw-http-requests-with-curl"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/110-controlling-appium-via-raw-http-requests-with-curl","slug":"110-controlling-appium-via-raw-http-requests-with-curl","path":"/editions/110-controlling-appium-via-raw-http-requests-with-curl","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0109.md","num":109,"rawMd":"\n\n

\n \"Automating\n

\n\niPad Pros run a slightly different version of iOS called 'iPadOS', and this version of iOS comes with several really useful features. Probably my favorite is the ability to run two apps side-by-side. This is called [Split View Multitasking](https://support.apple.com/en-ca/HT207582) by Apple, and getting it going involves a fair bit of gestural control for the user. Here's how a user would turn on Split View:\n\n1. Open the first app they want to work with\n2. Show the multitasking dock (using a slow short swipe up from the bottom edge)\n3. Touch, hold, and drag the icon of the second app they want to work with to the right edge of the screen\n\nFrom this point on, the two apps will be conjoined in Split View until the user drags the app separator all the way to the edge of the screen, turning Split View off. Of course, both apps must be designed to support split view for this to work. Let's now discuss how we can walk through this same series of steps with Appium to get ourselves into Split View mode, and further be able to automate whichever app of the two we desire. Unfortunately, there's no single command to make this happen, and we have to use a lot of tricky techniques to mirror the appropriate user behavior. Basically, we need to worry about these things:\n\n1. Ensuring both apps have been opened recently enough to show up in the dock\n2. Executing the correct gestures to show the dock and drag the app icon to trigger Split View\n3. Telling Appium which of the apps in the Split View we want to work with at any given moment\n4. Dragging the split to the side when we want to close out Split View\n\nIn this article, we're going to describe how to achieve steps 1-3, and I'll leave step 4 as an exercise for you (or as the subject of a future post).\n\n### Ensuring apps are in the dock\n\nFor our strategy to work, we need the icon of the app we want to open in Split View in the dock. The best way to make this happen is to ensure that it has been launched recently--in fact, *most* recently apart from the currently-running app. Let's take a look at the setup for an example where we'll load up both Reminders and Photos in Split View. In our case, we'll want Reminders on the left and Photos on the right. Because we're going to open up Photos on the right, we'll actually launch it *first* in our test, so that we can close it down, open up Reminders, and *then* open up Photos as the second app.\n\n```java\nDesiredCapabilities capabilities = new DesiredCapabilities();\ncapabilities.setCapability(\"platformName\", \"iOS\");\ncapabilities.setCapability(\"platformVersion\", \"13.3\");\ncapabilities.setCapability(\"deviceName\", \"iPad Pro (12.9-inch) (3rd generation)\");\ncapabilities.setCapability(\"app\", PHOTOS);\ncapabilities.setCapability(\"simulatorTracePointer\", true);\ndriver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\nwait = new WebDriverWait(driver, 10);\nsize = driver.manage().window().getSize();\n```\n\nIn this `setUp` method, we also construct a WebDriverWait, and store the screen dimensions on a member field, because we'll end up using them frequently. When we begin our test, the Photos app will be open. What we want to do next is actually terminate Photos, and launch Reminders. At this point, we've launched both the apps we want to work with, so they are both the most recently-launched apps, and will both show up in the recent apps section of the dock. Then, we go back to the Home Screen, so that the dock is visible:\n\n```java\n// terminate photos and launch reminders to make sure they're both the most recently-\n// launched apps\ndriver.executeScript(\"mobile: terminateApp\", ImmutableMap.of(\"bundleId\", PHOTOS));\ndriver.executeScript(\"mobile: launchApp\", ImmutableMap.of(\"bundleId\", REMINDERS));\n\n// go to the home screen so we have access to the dock icons\nImmutableMap pressHome = ImmutableMap.of(\"name\", \"home\");\ndriver.executeScript(\"mobile: pressButton\", pressHome);\n```\n\nThe reason we need to do this is one of the tricky points of this whole exercise. If an app is open, then the icons on the dock are not accessible to Appium for automation. However, on the Home Screen, the dock icons *are* accessible. What we want to do is take advantage of the fact that Photos and Reminders have both been opened recently, so we know their icons will be in the dock. Furthermore, we know that, as long as we don't open up any other apps, their icons will be in the *same spot* when we eventually show the dock later. Because we know they'll be in the same spot, we'll be able to use the position of the Photos dock icon on the Home Screen as a guarantee of where the Photos dock icon will be when we have Reminders open.\n\nIn the next stage of this flow, we figure out where the Photos icon is, and save that information for later. Then we re-launch Reminders, so that it is active and ready to share the screen with Photos.\n\n```java\n// save the location of the icons in the dock so we know where they are when we need\n// to drag them later, but no longer have access to them as elements\nRectangle photosIconRect = getDockIconRect(\"Photos\");\n\n// relaunch reminders\ndriver.executeScript(\"mobile: launchApp\", ImmutableMap.of(\"bundleId\", REMINDERS));\n```\n\nThere is an interesting helper method here. `getDockIconRect` just takes an app name, and returns the position of its dock icon in the screen:\n\n```java\nprotected Rectangle getDockIconRect(String appName) {\n By iconLocator = By.xpath(\"//*[@name='Multitasking Dock']//*[@name='\" + appName + \"']\");\n WebElement icon = wait.until(\n ExpectedConditions.presenceOfElementLocated(iconLocator));\n return icon.getRect();\n}\n```\n\nHere we use an xpath query to ensure that the element we retrieve is actually the dock icon and not the home screen icon. Then, we return the screen rectangle representing that element, so that we can use it later.\n\n### Showing the dock and entering Split View\n\nAt this point we are ready to call a special helper method designed to slowly drag the dock up in preparation for running the Split View gesture:\n\n```java\n// pull the dock up so we can see the recent icons, and give it time to settle\nshowDock();\nThread.sleep(800);\n```\n\nBasically, we show the dock, then wait a bit for it to cool down and settle in its place. What's the implementation we're working with?\n\n```java\nprotected void showDock() {\n swipe(0.5, 1.0, 0.5, 0.92, Duration.ofMillis(1000));\n}\n```\n\nWell, hello! It's our old friend `swipe` from [Edition 107](https://appiumpro.com/editions/107)! We're re-using it here to perform a slow swipe from the middle bottom of the screen, up just far enough to show the dock. Now that the dock is shown, we can actually enter Split View. To do that, we make use of a special iOS-specific method `mobile: dragFromToForDuration`, which enables us to perform a touch-and-hold on the location of the Photos dock icon, then drag it to the right side of the screen. We wrap this up in a helper method called `dragElement`. The usage and implementation are as follows:\n\n```java\n// now we can drag the photos app icon over to the right edge to enter split view\n// also give it a bit of time to settle\ndragElement(photosIconRect, 1.0, 0.5, Duration.ofMillis(1500));\nThread.sleep(800);\n```\n\n```java\nprotected void dragElement(Rectangle elRect, double endXPct, double endYPct, Duration duration) {\n Point start = new Point((int)(elRect.x + elRect.width / 2), (int)(elRect.y + elRect.height / 2));\n Point end = new Point((int)(size.width * endXPct), (int)(size.height * endYPct));\n driver.executeScript(\"mobile: dragFromToForDuration\", ImmutableMap.of(\n \"fromX\", start.x, \"fromY\", start.y,\n \"toX\", end.x, \"toY\", end.y,\n \"duration\", duration.toMillis() / 1000.0\n ));\n}\n```\n\nEssentially, we take the rect of a dock icon, pass in the ending x and y coordinate percentages, and the duration of the \"hold\" portion of the gesture. The `dragElement` helper converts these to the appropriate coordinates, and calls the `mobile:` method.\n\n### Working with simultaneously-open apps\n\nAt this stage in our flow, we've got both apps open in Split View! But if we take a look at the page source, we'll find that we only see the elements for *one* of the apps. And in fact, we can only work with one app's elements at a time. We can, however, tell Appium *which* app we want to work with, by updating the `defaultActiveApplication` setting to the bundle ID of whichever app you want to work with:\n\n```java\ndriver.setSetting(\"defaultActiveApplication\", PHOTOS);\nwait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId(\"All Photos\")));\ndriver.setSetting(\"defaultActiveApplication\", REMINDERS);\nwait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId(\"New Reminder\")));\n```\n\nIn the code above, you can see how we call `driver.setSetting`, with the appropriate setting name and bundle ID. After doing this for a given app, we can find elements within that app, and of course we can switch to any other app if we want as well.\n\nSo that's how we can enter into Split View and automate each app on the screen! Don't forget to check out the [full code example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition109_iPadOS_Split_Screen.java) for the full context.\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Working with iPadOS Multitasking Split View","shortDesc":"One of the greatest features of iPadOS is the ability to run multiple apps at once on the same screen, for a more powerful set of workflows. Using Appium and a bit of thoughtful hackery, we can mimic the user behaviors needed to enter into this Split View Multitasking mode.","liveAt":"2020-03-19 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-ipados-multitasking-split-view"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/109-working-with-ipados-multitasking-split-view","slug":"109-working-with-ipados-multitasking-split-view","path":"/editions/109-working-with-ipados-multitasking-split-view","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0108.md","num":108,"rawMd":"\n\n

\n \"Home\n

\n\niOS apps can register special shortcut actions that the app users can take from the home screen app icon itself. These actions are called [Home Screen Quick Actions](https://developer.apple.com/design/human-interface-guidelines/ios/extensions/home-screen-actions/), and provide a handy way to jump straight into a task in an iOS app without navigating through the initial app views.\n\nThe way that a user accesses these actions is by performing a \"tap and hold\" on the app icon, and then choosing one of the options from the context menu which appears. With Appium's XCUITest driver, nothing prevents us from automating the home screen itself, so we can easily open up these shortcut actions with Appium, the same way a user would. Thankfully, iOS puts helpful accessibility IDs on the home screen icons as well as the action buttons themselves. Let's walk through an example of how we'd implement the flow shown in the animated gif above: triggering the \"New Reminder\" action of the Reminders app.\n\nFirst, we need an Appium session. It doesn't really matter what app we choose to launch, since we're going to immediately close it and get to the home screen. We could choose, for example, to launch the Reminders app, to remain thematically consistent:\n\n```java\nDesiredCapabilities capabilities = new DesiredCapabilities();\ncapabilities.setCapability(\"platformName\", \"iOS\");\ncapabilities.setCapability(\"platformVersion\", \"13.3\");\ncapabilities.setCapability(\"deviceName\", \"iPhone 11\");\ncapabilities.setCapability(\"app\", \"com.apple.reminders\");\ndriver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n```\n\nNow, we need to get back to the home screen. There are a few ways to do this, but I chose the way which should be most bullet-proof, which is the `pressButton` mobile method:\n\n```java\nImmutableMap pressHome = ImmutableMap.of(\"name\", \"home\");\ndriver.executeScript(\"mobile: pressButton\", pressHome);\ndriver.executeScript(\"mobile: pressButton\", pressHome);\n```\n\nI call this method twice, because we're not necessarily guaranteed to end up on the *first* page of the home screen. To be more intelligent about it, we'd probably want to put this in a loop and simply call it until we wind up on the screen with the Reminders app (because maybe the user put it on a different screen, as well). Next, we find the icon itself and tap and hold it:\n\n```java\n// find the reminders icon and long-press it\nBy REMINDER_ICON = By.xpath(\"//*[@name='Home screen icons']//*[@name='Reminders']\");\nWebElement homeIcon = wait.until(ExpectedConditions.presenceOfElementLocated(REMINDER_ICON));\ndriver.executeScript(\"mobile: touchAndHold\", ImmutableMap.of(\n \"element\", ((RemoteWebElement)homeIcon).getId(),\n \"duration\", 2.0\n));\n```\n\nAs you can see, rather than constructing a typical Action sequence for this, we take advantage of the fact that we know we're on iOS, and use the iOS-specific `mobile: touchAndHold` method, to make sure that the OS recognizes our intent. Notice also that the icon we've chosen is qualified via a more complex XPath query. This is because in the case of iPadOS, we might have two instances of the icon on the screen---one in the main area, and one in the multitasking dock.\n\n```java\nBy ADD_REMINDER = MobileBy.AccessibilityId(\"New in Reminders\");\nBy LISTS = MobileBy.AccessibilityId(\"Lists\");\n\n// now find the home action using the appropriate accessibility id\nwait.until(ExpectedConditions.presenceOfElementLocated(ADD_REMINDER)).click();\n\n// prove we opened up the reminders app to the view where we can add a reminder\nwait.until(ExpectedConditions.presenceOfElementLocated(LISTS));\n```\n\nIn the last section of our test, above, we actually tap on the quick action text which was uncovered by our long tap. Then, we look for an element in the Reminders app that we know only shows up when we've gotten directly to the action (this is a screen which is not the typical initial screen of the app).\n\nThat's it! Nothing magic to it--just using Appium to script out exactly the behavior of a regular old human user.\n\nAs always, you can check out the code for this example in the context of a working project at the [Appium Pro repo](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition108_iOS_Home_Screen_Actions.java).\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Working with iOS App Home Screen Actions","shortDesc":"iOS apps have the ability to give the user access to shortcut actions that can be taken from the home screen. In this article, we take a look at how to automate that user flow using Appium.","liveAt":"2020-03-12 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-ios-app-home-screen-actions"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/108-working-with-ios-app-home-screen-actions","slug":"108-working-with-ios-app-home-screen-actions","path":"/editions/108-working-with-ios-app-home-screen-actions","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0107.md","num":107,"rawMd":"\n\n

\n \"Lists\n

\n\nOne common requirement for testing is to scroll a list until an element is found or some other condition is reached. Scrolling can be achieved in a variety of ways, but the most \"low-level\" way of doing so is using the Actions API. We've looked in depth at this API in the earlier edition on how to [automate complex gestures](https://appiumpro.com/editions/29). In that article, we looked at how to draw shapes! But what about something simpler, like scrolling a list?\n\nThe principles are all the same, but it's so common of an action it's a good idea to build some convenience methods for accomplishing it. That's what this article is about. Here are the requirements for our convenience methods:\n\n* We should be able to scroll a list view\n* We should be able to swipe in an arbitrary fashion, for an arbitrary amount of time\n* We should be able to define a swipe in screen-relative terms, not merely absolute pixels\n* Our actions should do the same thing on both iOS and Android\n* The methods should have equivalents with sane defaults\n\nLet's dive into how to do this!\n\n### The base class\n\nBecause we want to share this functionality between test classes, we make sure to put it on some kind of base class. Here's the base class I'll be using (sans imports; see the [full file](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition107_Base.java) on GitHub:\n\n```java\nabstract public class Edition107_Base {\n protected AppiumDriver driver;\n\n private By listView = MobileBy.AccessibilityId(\"List Demo\");\n private By firstCloud = MobileBy.AccessibilityId(\"Altocumulus\");\n private WebDriverWait wait;\n private Dimension windowSize;\n\n protected AppiumDriver getDriver() {\n return driver;\n }\n\n @After\n public void tearDown() {\n try {\n getDriver().quit();\n } catch (Exception ign) {}\n }\n\n protected void navToList() {\n wait = new WebDriverWait(getDriver(), 10);\n wait.until(ExpectedConditions.presenceOfElementLocated(listView)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(firstCloud));\n }\n}\n```\n\nSo far, the base class just takes care of navigating to the list view in our application as well as tearing down the app. Note that we have a function called `getDriver`, which will be overridden by the specific test cases and allow each test case to return its own driver (which might be an iOS driver or an Android driver). Now let's start building out our convenience methods.\n\n### Low-level swipe\n\nEverything we want to do (scrolling, swiping) can be conceived of at the root as an action with basically 4 steps:\n\n1. Move the finger to a location above the screen\n2. Touch the finger to the screen\n3. While touching the screen, move the finger to another location, taking a certain amount of time to do so\n4. Lift the finger off the screen\n\nEncoding these steps will be our first task, and will result in the method we build on top of for everything else. Let's walk through the implementation:\n\n```java\nprotected void swipe(Point start, Point end, Duration duration)\n```\n\nWe'll call this method 'swipe', and allow the user to define the start point, end point, and duration.\n\n```java\nAppiumDriver d = getDriver();\nboolean isAndroid = d instanceof AndroidDriver;\n```\n\nTo perform the swipe, we need the driver instance, which we get, and we also give ourselves a handy way of referring to whether the driver is iOS or Android.\n\n```java\nPointerInput input = new PointerInput(Kind.TOUCH, \"finger1\");\nSequence swipe = new Sequence(input, 0);\nswipe.addAction(input.createPointerMove(Duration.ZERO, Origin.viewport(), start.x, start.y));\nswipe.addAction(input.createPointerDown(MouseButton.LEFT.asArg()));\n```\n\nIn the section above, we begin to construct our action sequence. First of all we define our pointer input and the sequence object which will represent the swipe. The next two actions are common to both iOS and Android. We initially move the pointer to the start point (taking no explicit time duration to do so), and then lower the pointer to touch the screen. Now, we'll take a look at some code that changes a bit depending on platform:\n\n```java\nif (isAndroid) {\n duration = duration.dividedBy(ANDROID_SCROLL_DIVISOR);\n} else {\n swipe.addAction(new Pause(input, duration));\n duration = Duration.ZERO;\n}\nswipe.addAction(input.createPointerMove(duration, Origin.viewport(), end.x, end.y));\n```\n\nThis is the part of the function which is responsible for moving to the end point, with a certain duration. Here there is a quirk for each iOS and Android we have to worry about. For Android, I've noticed that the actual time taken by the action is always much greater than the amount of time we actually specify. This may not be true across the board, but I defined a static variable `ANDROID_SCROLL_DIVISOR` to represent this (the value we're using here is `3`).\n\nOn iOS, the duration of the pointer move is not actually encoded on the move action at all; it is encoded on a `pause` action which is inserted *before* the move action, which is why we are adding that action here if we're using iOS (before then setting the duration to zero for the actual move). This is a bit of an odd situation on iOS and I wouldn't be surprised if we can find a way to make sure these APIs are equivalent in the future.\n\n```java\nswipe.addAction(input.createPointerUp(MouseButton.LEFT.asArg()));\nd.perform(ImmutableList.of(swipe));\n```\n\nThe last bit of the method is above, where we finally lift the finger from the screen, and tell the driver to actually perform the sequence we've encoded.\n\n### Swiping relatively\n\nOur swipe method is great, but it requires absolute screen coordinates. What if we're running the same test on different devices with different screen dimensions? It would be much better if we could specify our swipe in terms relative to the height and width of the screen. Let's do that now.\n\n```java\nprivate Dimension getWindowSize() {\n if (windowSize == null) {\n windowSize = getDriver().manage().window().getSize();\n }\n return windowSize;\n}\n\nprotected void swipe(double startXPct, double startYPct, double endXPct, double endYPct, Duration duration) {\n Dimension size = getWindowSize();\n Point start = new Point((int)(size.width * startXPct), (int)(size.height * startYPct));\n Point end = new Point((int)(size.width * endXPct), (int)(size.height * endYPct));\n swipe(start, end, duration);\n}\n```\n\nHere we have defined a new version of `swipe`, that takes start and end values in percentages, not absolute terms. To make this work, we need another helper function which retrieves (and caches) the window size. Using the window size (height and width), we are able to calculate absolute coordinates for the swipe.\n\n### Scrolling\n\nNow we are in a good position to build up our scroll-related convenience methods. What is a scroll exactly? Well, it's technically a swipe where we don't care about one of the dimensions. If I'm scrolling a list down, it means I'm performing a slow upwards swipe action in reality, where the only change in motion that matters is change in the y-axis.\n\nConceptually, we have 4 directions we can scroll, so we can create an enum to help us define that:\n\n```java\npublic enum ScrollDirection {\n UP, DOWN, LEFT, RIGHT\n}\n```\n\nNow, we can construct a `scroll` method which takes one of these directions. It's also helpful to make the scroll *amount* configurable. Do we want a short scroll or a long scroll? What we can do is allow the user to define this again in terms relative to the screen height or width. So If I say I want a scroll amount of `1`, that means I should scroll the equivalent of a full screen. `0.5` would mean the equivalent of a half screen. Let's look at the implementation:\n\n```java\nprotected void scroll(ScrollDirection dir, double distance) {\n if (distance < 0 || distance > 1) {\n throw new Error(\"Scroll distance must be between 0 and 1\");\n }\n Dimension size = getWindowSize();\n Point midPoint = new Point((int)(size.width * 0.5), (int)(size.height * 0.5));\n int top = midPoint.y - (int)((size.height * distance) * 0.5);\n int bottom = midPoint.y + (int)((size.height * distance) * 0.5);\n int left = midPoint.x - (int)((size.width * distance) * 0.5);\n int right = midPoint.x + (int)((size.width * distance) * 0.5);\n if (dir == ScrollDirection.UP) {\n swipe(new Point(midPoint.x, top), new Point(midPoint.x, bottom), SCROLL_DUR);\n } else if (dir == ScrollDirection.DOWN) {\n swipe(new Point(midPoint.x, bottom), new Point(midPoint.x, top), SCROLL_DUR);\n } else if (dir == ScrollDirection.LEFT) {\n swipe(new Point(left, midPoint.y), new Point(right, midPoint.y), SCROLL_DUR);\n } else {\n swipe(new Point(right, midPoint.y), new Point(left, midPoint.y), SCROLL_DUR);\n }\n}\n```\n\nBasically what's going on her is that we're using the distance parameter to define where the possible start and end points of the swipe we're constructing should be. Conceptually, the start point of a scroll-swipe is going to be half the distance from the mid-point of the screen, along the appropriate axis. With all of these points defined, we can then simply start and end at the appropriate point corresponding to the direction we want to scroll!\n\n### Nice default scrolling\n\nIn a lot of cases, we don't need to specify the distance exactly, so we can rely on a sane default:\n\n```java\nprotected void scroll(ScrollDirection dir) {\n scroll(dir, SCROLL_RATIO);\n}\n```\n\n(In this project, the default distance is `0.8`, to make sure we don't accidentally scroll even a pixel past content we might care about)\n\nIn general, we usually care about scrolling lists *down*, so we can also have a super convenient method for doing that:\n\n```java\nprotected void scroll() {\n scroll(ScrollDirection.DOWN, SCROLL_RATIO);\n}\n```\n\nThat's it! That's the set of helper methods that tick all our requirement boxes above.\n\n### Putting it all together\n\nLet's take a look at a cross-platform example using these methods, basically scrolling down and then back up in a list:\n\n```java\n@Test\npublic void testGestures() {\n navToList();\n scroll();\n scroll(ScrollDirection.UP);\n}\n```\n\nThis method can be used in either an iOS or an Android test, and I've written one for each platform. You can review the full code on GitHub:\n\n* [iOS scroll test](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition107_Generic_Swipe_Actions_iOS.java)\n* [Android scroll test](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition107_Generic_Swipe_Actions_Android.java)\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Designing a Cross-Platform Swipe/Scroll Helper","shortDesc":"Making sure swiping and scrolling happens equivalently across platforms, and in a way which is super convenient to access from test cases, can be a challenge. In this article I walk through the process of building such a helper in Java.","liveAt":"2020-03-05 10:00","newsletterTopNote":"And a reminder: there's still time to register for the Appium Pro intermediate/advanced workshop taking place in a few weeks in London!","canonicalRef":"https://www.headspin.io/blog/designing-a-cross-platform-swipe-scroll-helper"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/107-designing-a-cross-platform-swipescroll-helper","slug":"107-designing-a-cross-platform-swipescroll-helper","path":"/editions/107-designing-a-cross-platform-swipescroll-helper","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0106.md","num":106,"rawMd":"\n\n

\n \"Notifications\"\n

\n\nFor most of the history of Appium, it has not been possible to inspect notifications received on an Android device, especially the \"status bar\" notifications you can only look at by pulling them down from the status bar. To automate verification of notifications received, the brute-force approach meant navigating to this notifications view and hoping to get lucky reading the text elements on that page. More recently, the Appium team figured out a way to read notifications via the [Appium Settings](https://github.com/appium/io.appium.settings) helper app. This is a little app which is installed alongside an Appium session, and facilitates device-level functionality that is not possible merely through UiAutomator2 or Espresso.\n\nBasically, the Appium Settings app can be granted the permission to read any incoming notifications. It can then save these notifications for retrieval by the main Appium process using ADB. All of this complexity is wrapped up in one Appium command (first released in Appium 1.16):\n\n```java\ndriver.executeScript(\"mobile: getNotifications\");\n```\n\nFor this command to work, the Appium Settings app must of course be granted notification read permissions. This can be done manually in the \"Notification Access\" area of your Android system settings (the precise way to navigate to this view differs based on device). Or, you could automate it using Appium as part of your device provisioning steps.\n\nAt the moment, the return value for this method is a bit complex. In JSON, it would look something like this:\n\n```json\n{\n \"statusBarNotifications\": [\n {\n \"isGroup\":false,\n \"packageName\":\"io.appium.settings\",\n \"isClearable\":false,\n \"isOngoing\":true,\n \"id\":1,\n \"tag\":null,\n \"notification\":{\n \"title\":null,\n \"bigTitle\":\"Appium Settings\",\n \"text\":null,\n \"bigText\":\"Keep this service running, so Appium for Android can properly interact with several system APIs\",\n \"tickerText\":null,\n \"subText\":null,\n \"infoText\":null,\n \"template\":\"android.app.Notification$BigTextStyle\"\n },\n \"userHandle\":0,\n \"groupKey\":\"0|io.appium.settings|1|null|10133\",\n \"overrideGroupKey\":null,\n \"postTime\":1576853518850,\n \"key\":\"0|io.appium.settings|1|null|10133\",\n \"isRemoved\":false\n }\n ]\n}\n```\n\nBut of course, in the Appium Java client, we will not get a JSON response, so it will be our job to cast the response appropriately:\n\n```java\nMap notifications = (Map)driver.executeScript(\"mobile: getNotifications\");\n```\n\nAnd we'll need to keep casting as we navigate through the object. Basically, we have a main key `statusBarNotifications`, which returns an array of notification objects. Each of these notification objects itself has a lot of metadata associated with it (you can use this metadata to filter by the package ID of your app, for example). The `notification` key of this object has the main content we're looking for, including the title and the text of the notification.\n\nThis technique is especially powerful because, having granted the special permission to Appium, you can read notifications for any app on the device.\n\nThat's it! Check out a full example where we walk through each notification received since Appium started, and print it out to the console, making sure we appropriately choose the type of notification text and title which has content:\n\n```java\npublic class Edition106_Android_Notifications {\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-v1.10.0.apk\";\n private AndroidDriver driver;\n\n @Before\n public void setUp() throws Exception {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", APP);\n\n driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n @SuppressWarnings(\"unchecked\")\n public void testNotifications() {\n Map res = (Map)driver.executeScript(\"mobile: getNotifications\");\n List> notifications = (List>)res.get(\"statusBarNotifications\");\n for (Map notification : notifications) {\n Map innerNotification = (Map)notification.get(\"notification\");\n if (innerNotification.get(\"bigTitle\") != null) {\n System.out.println(innerNotification.get(\"bigTitle\"));\n } else {\n System.out.println(innerNotification.get(\"title\"));\n }\n if (innerNotification.get(\"bigText\") != null) {\n System.out.println(innerNotification.get(\"bigText\"));\n } else {\n System.out.println(innerNotification.get(\"text\"));\n }\n }\n }\n}\n```\n\nAs always, you can view the full source within a context of a working project [at the Appium Pro repo](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition106_Android_Notifications.java).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Retrieving Status Bar Notifications for Android Devices","shortDesc":"Getting the notifications that various apps can leave on your system is a challenging task. Thankfully, Appium has found a way to communicate with its helper app to retrieve these notifications for you and return them to your test script along with a ton of potentially quite-useful metadata.","liveAt":"2020-02-19 10:00","canonicalRef":"https://www.headspin.io/blog/retrieving-status-bar-notifications-for-android-devices"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/106-retrieving-status-bar-notifications-for-android-devices","slug":"106-retrieving-status-bar-notifications-for-android-devices","path":"/editions/106-retrieving-status-bar-notifications-for-android-devices","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0105.md","num":105,"rawMd":"\n\nSo far in this series, we've looked at [mobile performance testing in general](https://appiumpro.com/editions/102), as well as [free tools for mobile performance testing](https://appiumpro.com/editions/103). Now it's time to discuss *paid* tools. Obviously, Appium is free and open source, and it's nice to focus on and promote free tools. But at the end of the day, sometimes paid tools and services can be valuable, especially in a business setting where the budget is less important than the end result. I want to make sure you have some decent and reliable information about the whole ecosystem, which includes paid tools, so will from time to time let you know my thoughts on this as well!\n\n> Disclaimers: This is not intended to be completely comprehensive, and even for the services I cover, I will not attempt to give a full overview of all their features. The point here is to drill down specifically into performance testing options. Some services might not be very complete when it comes to their performance testing offering, but that doesn't mean they don't have their strong points in other areas. For each tool, I'll also note whether I've personally used it (and am therefore speaking from experience) or whether I'm merely using their publicly-available marketing information or docs as research. Finally, I'll note for transparency that HeadSpin is an Appium Pro sponsor, and in the past I worked for Sauce Labs.\n\n## Appium Cloud Services with Performance Features\n\nMost interesting for us as Appium users will be services that allow the collection of performance data at the same time as you run your Appium tests. This is in my opinion the 'holy grail' of performance testing--when you don't have to do anything in particular to get your performance test data! In my research I found four companies that in one way or another touch on some of the performance testing topics we've discussed as part of their Appium-related products.\n\n### [HeadSpin](https://ui.headspin.io/register?referral=start-testing-appiumpro)\n\n> I've used this service\n\nHeadSpin is the most performance-focused of all the service providers I've seen. Indeed, they began life with performance testing as their sole mission, so this is not surprising. Along the way, they added support for Appium and have been heavy contributors to Appium itself in recent years. HeadSpin supports real iOS and Android devices in various geographical locations around the world, connected to a variety of networks.\n\n

\n \"HeadSpin\"\n

\n\nWhen you run any Appium test on HeadSpin, you get access to a report that looks something like the one here. Basically, HeadSpin tracks a ton of different metrics in what they call a [performance session report](https://ui.headspin.io/docs/performance-session-report). The linked doc highlights some of the metrics captured:\n\n> network traffic (in HAR and PCAP format), device metrics including CPU, memory, battery, signal strength, screen rotation, frame rate, and others, device logs, client-side method profiling, and contextual information such as Appium commands issued during the test, what app was in the foreground, and other contextual metadata.\n\nOn top of this, they try to show warnings or make suggestions based on industry-wide targets for various metrics. And this is all without you have to instrument your app or make specific commands to retrieve specific metrics. HeadSpin covers the most metrics out of any product or company I'm familiar with, including really difficult-to-measure quantities like the MOS (mean opinion score) of a video as a proxy for video quality.\n\nOne other feature useful for performance testing is what they call the 'HeadSpin bridge', which allows bridging remote devices to your local machine via their `hs` CLI tool. Concretely, this gives you the ability to run `adb` on the remote device as if it were local, so any Android-related performance testing tools that work on a local device will also work on their cloud devices, for example the [CPU Profiler](https://developer.android.com/studio/profile/cpu-profiler). The bridge works similarly for iOS, allowing step debugging or [Instruments profiling](https://developer.apple.com/library/archive/documentation/AnalysisTools/Conceptual/instruments_help-collection/Chapter/Chapter.html) on remote devices, as if they were local.\n\nIn terms of network conditioning, HeadSpin offers a number of options that can be set on a per-test basis, including network shaping, DNS spoofing, header injection (which is super useful for tracing network calls all the way through backend infrastructure), etc... This is in addition to requesting devices that exist in certain geographical regions.\n\nFinally, it's worth pointing out that HeadSpin tries to make it easy for teams to use the performance data captured on their service by spinning up a custom \"Replica DB\" for each team, where performance data is automatically written. This \"open data\" approach means it's easy to build custom dashboards, etc..., on top of the data without having to pre-process it or host it yourself.\n\n### [Sauce Labs](https://saucelabs.com)\n\n> I've used this service\n\nHistorically, Sauce Labs was focused primarily on cross-browser web testing, and so it's no surprise they have some performance-related information available about their web browser sessions. That's not what we're considering today, however, but rather Sauce's mobile offering. They have a virtual device cloud of iOS simulators and emulators, but as we mentioned before, performance testing of virtual devices is not particularly meaningful.\n\nSauce's real device cloud supports a feature called [Device Vitals](https://wiki.saucelabs.com/display/DOCS/Device+Vitals+for+Mobile+Applications) which provides a broad set of performance metrics, including CPU, memory, and performance information. It works with live/manual testing, but is also supported via their Appium interface when a special capability `recordDeviceVitals` is set. At this stage of the product it appears the only output for Appium tests is a CSV file which can be downloaded, meaning the user would be responsible for extracting and displaying the performance data itself. However, the live/manual usage shows a graph of performance data in real time:\n\n

\n \"Sauce\n

\n\nSauce also supports a virtual device bridge, which they call [Virtual USB](https://wiki.saucelabs.com/display/DOCS/Testing+with+Virtual+USB+on+Real+Devices). It is generally available for Android and in beta for iOS, and works more or less the same as HeadSpin's bridge feature.\n\nIn my experience, and as far as I can tell, Sauce does not give users the ability to specify network conditions for device connections, so testing app performance in a variety of lab conditions is not possible. However, it is possible to direct Sauce devices to use a proxy connection, including something like Charles Proxy (see below).\n\n### [AWS Device Farm](https://aws.amazon.com/device-farm/)\n\n> I've used this service\n\nAWS has its own Appium cloud, which works on a bit of a different model than HeadSpin or Sauce. With AWS, you compile your whole testsuite into a bundle and send it off to Device Farm, which then unpacks it and uses it to run tests locally. Theoretically, this gives you a lot of flexibility about how you can connect to a device during the test, and it's probably possible (though I haven't tested this) to run some of your own performance testing diagnostics on the device under test, even as your Appium script is going through its paces.\n\nLuckily, you may not need to worry about this because Device Farm does capture some performance information as your test is executing, namely CPU, memory usage, and thread count over time:\n\n

\n \"Device\n

\n\nDevice Farm does allow you to [simulate specific network conditions](https://docs.aws.amazon.com/devicefarm/latest/developerguide/how-to-simulate-network-connections-and-conditions.html) by creating a 'network profile' (that acts basically the same as the profiles we discussed in the article on [simulating network conditions for virtual devices](https://appiumpro.com/editions/104)\n\n### [Experitest](https://experitest.com)\n\n> Information retrieved from company website\n\nExperitest is a mobile testing company with Appium support, who also advertise performance data collection. They say they support different network conditions and geographical locations for their devices, as well as performance metrics like:\n\n> * Transaction duration\n> * Speed Index\n> * CPU consumption\n> * Memory Consumption\n> * Battery Consumption\n> * Network data (upload and download)\n\nI am not entirely sure what a 'transaction' is, but it appears to be a user-definable span of time or sequence of app events.\n\n

\n \"Experitest\n

\n\nI could not find more specific information in the docs about how this data is captured, and whether it is done so automatically as a part of an Appium test or whether it is a separate feature/interface. The best I can figure out, during test recording the users define start/stop points for performance 'transaction' recording, though this does not explain whether it is possible with Appium tests that you have brought yourself and did not develop in Experitest's 'Appium Studio' product.\n\nLike AWS Device Farm, Experitest apparently has the ability to manually create network 'profiles' that can be applied to certain tests or projects.\n\n### [Perfecto Mobile](https://perfectomobile.com)\n\n> Information retrieved from company website\n\nPerfecto Mobile is probably the oldest company in the entire automated mobile testing space, and they've had a lot of time to build out their support for performance testing. It looks like Perfect focuses on network condition simulation via their concept of [Personas](https://developers.perfectomobile.com/display/PD/Built-in+personas), which are essentially the same types of network profiles we have seen in other places.\n\nYou can set the Persona you want your Appium test to use via a special desired capability. Given my perusal of the docs, it appears possible to set \"points of interest\" in your test, that you can use for verification later on. It wasn't clear, but it's possible that setting one of these \"points of interest\" might capture some specific performance data at that time for later review, however I couldn't find any specific mention of the types of performance metrics that Perfecto captures, if any.\n\nIn addition, I'm not quite sure what \"Appium\" means exactly for Perfecto. From their [Appium landing page](https://www.perfecto.io/integrations/appium), we see this message:\n\n> Appium testing has limitations that may prevent it from being truly enterprise ready. Perfecto has re-implemented the Appium server so it can deliver stronger automated testing controls, mimic real user environments, and support the unique needs of today’s Global 1000 companies.\n\nWhat this implies is that, if you're using Perfecto, you're not actually running the Appium open source project code. Whether that is a good or bad thing is up to you to decide, however I can only imagine that it could generate confusion as their internal reimplementation inevitably drifts from what the Appium community is doing! How easy is it to migrate Appium tests running on Perfecto to another cloud? If they're not using standard open source Appium servers, this could be a difficult question to answer. I'm a big fan of what Perfecto is trying to do with their product, but (as is probably not surprising given my role with the Appium project) I don't think it's beneficial for any cloud provider to \"reimplement\" Appium.\n\n## Other Products\n\nCloud services are not the only types of paid products used for mobile performance testing. We also have SDKs (which are built into the app under test), and standalone software used for various purposes. None of these have any special relationship with or support with Appium, so you're responsible for figuring out how to make them work in conjunction with your Appium test.\n\n### [Charles Proxy](https://www.charlesproxy.com/)\n\n> Information retrieved from company website\n\nCharles Proxy is a networking debugging proxy that has special support for mobile devices. You can use it to inspect or adjust network traffic coming into or out of your devices. Charles comes with special VPN profiles you can install on your devices to get around issues with trusting Charles's SSL certificates, etc...\n\nCharles's main features are: SSL proxying, bandwidth throttling (network conditioning), request replay, and request validation.\n\n### [Firebase](https://firebase.google.com/)\n\n> Information retrieved from company website\n\nGoogle's Firebase is a *lot* of things, apparently intended to be a sort of one-stop-shop especially for Android developers. It's got databases and auth services, analytics and distribution tools. Firebase doesn't have any officially supported Appium product, though they do support automated functional testing of Android apps (via Espresso and UiAutomator 2) and iOS apps (via XCTest) through their Test Lab product.\n\nOf more interest for us today is their [performance monitoring](https://firebase.google.com/products/performance/) product. The way this works is by bundling in a particular SDK with your app which enables in-depth performance tracing viewable in the Firebase console. You start and stop traces using code that leverages the SDK in test versions of your app, and that's it. Of course, this means modifying your app under test, and I couldn't find any discussion of whether performance or network data is captured \"from the outside\" during test runs in the Firebase Test Lab.\n\n### [New Relic](https://newrelic.com/)\n\n> Information retrieved from company website\n\nAnother SDK-based option is [New Relic Mobile](https://newrelic.com/products/mobile-monitoring/features). Embedding their SDK in your app enables tracking of a host of network and other performance data. Where New Relic appears to have an advantage is in showing the connections your app has with other New Relic-monitored services (backend sites, for example).\n\n

\n \"New\n

\n\nOtherwise, New Relic provides most of the same types of performance data you'd expect. They also give you a \"User Interaction Summary\" which appears to be similar to HeadSpin's high level detail of potential UX issues. In addition, metrics can be viewed in aggregate across device types, so it's possible to easily spot performance trends on the level of device type as well.\n\nSince this is an SDK-based solution, you'll have to modify your app under test to gather the data. Presumably you could then use it in your Appium tests just fine!\n\n## Conclusion\n\nIn this article we examined a number of cloud services and other types of paid software used for performance testing. A small handful of these companies supports Appium as a first-class citizen, meaning you can use their cloud to run your Appium tests themselves, and potentially get a lot of useful performance data for free along with it.\n\nIn my mind, this is the best possible approach for Appium users--just write your functional tests the way a user would walk through your app, then run the tests on one of these platforms, and you'll get performance reports for free! (Or rather, for hopefully no *additional* fee). The best kind of performance test is obviously the one you didn't have to write explicitly yourself, but which still found useful information that can easily be acted on in order to improve the overall UX of your app.\n\nOh, and did I miss a good candidate for this article? It's entirely possible I did! This is a big and growing industry. Let me know about it and I'll update this list if I find a relevant product.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Paid Tools And Services For Mobile App Performance Testing","shortDesc":"We wrap up our long series on mobile performance and UX testing with a look at some of the non-free tools or services available to assist with the task. Some of these services provide a way to get performance reports for free as a result of an Appium test. Other tools are standalone products that can be used alongside Appium, but might require some extra setup on your part.","liveAt":"2020-02-05 10:00","canonicalRef":"https://www.headspin.io/blog/an-appium-cloud-service-with-performance-features"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/105-paid-tools-and-services-for-mobile-app-performance-testing","slug":"105-paid-tools-and-services-for-mobile-app-performance-testing","path":"/editions/105-paid-tools-and-services-for-mobile-app-performance-testing","news":{"rawMd":"Apologies that this newsletter is a day late! I wanted to be sure to do enough research on all the companies I could find in order to put together a useful set of descriptions. Also, I've been hard at work on the upcoming [Appium Pro Workshops](https://appiumpro.com/workshops). For those of you who are planning on attending--get ready, we're going to have some fun!\n","num":105,"mdPath":"/vercel/path0/content/news/0105.md"},"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0103.md","num":103,"rawMd":"\n\n

\n \"Wireshark\"\n

\n\nIn a previous edition of Appium Pro, I discussed [mobile performance and UX testing](https://appiumpro.com/editions/102) at a conceptual level. In this edition, let's take a look at some pointers for tracking the various metrics we saw earlier. There are way too many metric to consider each in detail, so we'll just outline a solution in each case; stay tuned for future articles with fully-baked examples (and let me know which metrics you're most interested in reading about!) In this set of pointers, I'm going to focus on free and open source approaches, saving a deeper look at paid services for a follow-up article.\n\n### Time-to-interactivity (TTI)\n\nThis metric measures the time elapsed from when the user launches your app till it's actually usable. A first-pass solution to measuring TTI would be throwing some timestamps into your testcode before starting an Appium session and after the session is fully started. However, this approach falls short in several ways:\n\n* You're measuring Appium startup time in addition to app startup time\n* Just because you have an Appium session loaded doesn't mean your app is actually interactable yet.\n\nAs a second pass, we can try an approach like we did in [Edition 97](https://appiumpro.com/editions/97), where we delayed actually launching the app until after Appium startup itself was complete. This gets us a lot closer to a solid metric, but is still not bulletproof, because measurements are contingent on a find-element poll, which might not occur at exactly the moment the app is fully loaded.\n\n### DNS resolution\n\nHow on earth can you test that your mobile app API calls are hitting the appropriate server? Obviously checking this assumes that (a) you know the domains the app is trying to hit, and (b) you know the IP addresses it's *supposed* to resolve to (for its region, or whatever). The only obvious and free solution I came up with to this problem also assumes (c) you are running on emulators and simulators (i.e., using the same network as your test machine).\nWhen you're running emulators and simulators, the network used by the device under test is in fact your host machine's network, and you can use [tcpdump](https://www.tcpdump.org/) to examine all network traffic leaving your machine, including DNS resolution requests. Of course, looking at all TCP traffic will be a bit of a firehose, so you can restrict `tcpdump`'s output to requests to nameservers (typically running on port 53):\n\n```\nsudo tcpdump udp port 53\n```\n\nThe output of DNS requests will look something like:\n\n```\n20:09:54.099145 IP 10.252.121.60.62613 > resolver.qwest.net.domain: 52745+ A? play.google.com. (33)\n20:09:54.101718 IP resolver.qwest.net.domain > 10.252.121.60.62613: 52745 1/0/0 A 216.58.217.46 (49)\n```\n\nSo it would be straightforward (if not easy) to write a script that runs `tcpdump` as a subprocess, scrapes output during the space of a mobile app call, and makes assertions about domain name resolution!\n\nUnfortunately, this would not work on real devices. For real devices you'd need to set up some kind of DNS proxy and have your devices configured to use it for DNS resolution, so you can log or track data about DNS requests.\n\n### Time till 1st byte, TCP delay, TLS handshake time\n\nThese low-level network metrics are best captured by a network analysis tool like [Wireshark](https://wireshark.org), which can listen to all TCP traffic that passes through a network interface, and provide more data than a simple `tcpdump` log. The data Wireshark captures is made available in a special `.pcap` file, which has become a standard for representing network data.\n\nNote that this means you could also use Wireshark for DNS request examination!\n\nUsing a `.pcap` file (which stands for \"packet capture\" because it represents every single packet that traverses a network interface), you can get a ton of information about network requests, including the time they took. `.pcap` files can be parsed in pretty much any language, for example with the [jnetpcap](https://mvnrepository.com/artifact/jnetpcap/jnetpcap) library for Java. Parsing the file enables you to determine all kinds of facts about network requests, for example how long they took.\n\nWireshark is designed as a GUI tool, which limits is utility in CI environments, though it can be launched from the CLI in an auto-record mode, and directed to write the resulting capture file to any location.\n\nAnd finally, any method like this will again only work on virtual devices, unless you're somehow able to route real device traffic through a router which is itself running Wireshark!\n\n### Other network analysis approaches\n\nSo far I've tried to recommend very low-level and general tools, but there are also some other options. Some of these are specific to Android:\n\n* [Advanced Network Profiling](https://developer.android.com/studio/profile/android-profiler) - a library produced by Google that you build into your app, which can capture and output network data\n* [Stetho](https://facebook.github.io/stetho/) - a similar library from Facebook that provides a web devtools-like environment for inspecting your app's network requests. Also requires building into the app.\n\nCross-platform solutions would include the use of man-in-the-middle proxies:\n\n* [Postman](https://www.getpostman.com/) - a GUI-based tool that can provide network analysis information when you proxy mobile app requests through it.\n* [mitmproxy](https://mitmproxy.org/) - an open-source CLI-based tool which allows proxying of HTTP/HTTPS traffic\n\nProxy solutions require configuring the device under test with the correct proxy settings, and trusting the proxy as a certificate authority for HTTPS requests to be visible for analysis. To learn more about how to put all this together, you can check out the Appium Pro articles on [capturing iOS simulator network traffic](https://appiumpro.com/editions/62), [capturing Android emulator traffic](https://appiumpro.com/editions/63), and [capturing network traffic with Java in your test scripts](https://appiumpro.com/editions/65).\n\n### CPU utilization, memory consumption, etc...\n\nThese low-level device metrics can be retrieved in a minimal way using Appium itself. Check out these earlier Appium Pro articles for more info (note that with iOS especially, programmatically using retrieved performance info is not straightforward at all):\n\n* [Capturing performance data for Android apps](https://appiumpro.com/editions/5)\n* [Capturing performance data for iOS apps](https://appiumpro.com/editions/12)\n\n### Video quality, blank screen time, animation time\n\nThese last metrics are particularly hard to determine using free and open source tools. Video quality is a subjective measurement and to determine it in an automated fashion would require (a) the ability to capture video from the screen (which Appium can do via its [startScreenRecording](https://appiumpro.com/editions/82) command), and (b) some machine learning model which could take a video as input and provide a quality judgment as output.\n\nI could not find a good open source tool to measure video quality (though there is a paper [outlining how such a tool might be developed](https://www.researchgate.net/publication/301464319_An_Open_Source_Platform_for_Perceived_Video_Quality_Evaluation)).\n\nSimilarly, for blank screen time, the best approach would involve training some kind of machine learning model to recognize screens with no elements, though an easy approximation could be made by using Appium and logging how much time elapses with a typical element wait.\n\n### Conclusion\n\nIn general, the world of free and open source network and performance testing is a bit of a rough and tumble place. In some cases, the tools exist, but are not well-integrated with the ecosystem a tester is used to working with. In other cases, we are limited to certain platforms or devices. In yet other cases, the best tools out there today are probably not free and open source.\n\nFor that reason, stay tuned, because we're going to consider some paid tools and services in a future edition of Appium Pro, so you have the best understanding of the landscape and can make good decisions about the costs and benefits of adopting various approaches to performance testing of mobile apps.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Free Tools For Mobile App Performance Testing With Appium","shortDesc":"Having already looked at network and performance testing from a high level, in this article we dive into specific tools and techniques for capturing some of the most relevant performance and UX metrics for mobile apps.","liveAt":"2020-01-22 10:00","canonicalRef":"https://www.headspin.io/blog/free-tools-for-mobile-app-performance-testing-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/103-free-tools-for-mobile-app-performance-testing-with-appium","slug":"103-free-tools-for-mobile-app-performance-testing-with-appium","path":"/editions/103-free-tools-for-mobile-app-performance-testing-with-appium","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0102.md","num":102,"rawMd":"\n\nWhat is mobile app performance testing, exactly? What should we care about in terms of app performance and how can we test app performance as part of our automated testsuite? This is a topic I've been thinking a lot about recently, and so I convinced Brien Colwell (CTO of [HeadSpin](https://www.headspin.io/audio-visual-platform)) to sit down with me so I could pester him with some questions. I took our conversation and turned it into a sort of FAQ of mobile app performance testing. None of this is specifically tied to Appium--stay tuned for future editions where we look at how to achieve some of the goals discussed in this article, using Appium by itself or in conjunction with other tools and services.\n\n### What is performance testing all about?\n\nReally, we could think of performance testing as a big part of a broader concept: UX testing (or User Experience Testing). The idea here is that a user's experience of your app goes beyond the input/output functionality of your app. It goes beyond a lot of the things we normally associate with Appium (though of course it includes all those things too--an app that doesn't work does not provide a good experience!) It reflects the state of the app market, where the world is so crowded with nearly-identical apps, that small improvements in UX can mean the difference between life and death for one of these startups.\n\nYears ago, I considered performance testing to be exclusively in the domain of metrics like CPU or memory usage. You only did performance testing when you wanted to be sure your app had no memory leaks, that kind of thing. And this is still an important part of performance testing. But more broadly, performance testing now focuses on attributes or behaviors of your application as they interface with psychological facts about the user, like how long they are prepared to wait for a view to load before moving on to another app. One quick way of summarizing all of this is to define performance testing as ensuring that your app is responsive to the directions of the user, across all dimensions.\n\n### What performance metrics should we care most about?\n\nIt's true that classic specters of poor software performance, like memory leaks or spinning CPUs, can plague mobile app experience. And there is good reason to measure and profile these metrics. However, the primary cause of a bad user experience these days tends to be network-related. So much of the mobile app experience is dominated by the requirement of loading data over the network, that any inefficiency there can cause the user to experience painfully frustrating delays. Also, testing metrics like memory or CPU usage can often be adequately accomplished locally during development, whereas network metrics need to be tested in a variety of conditions in the field.\n\nTo this end, we might track metrics like the following:\n\n* DNS resolution (did the name resolve to an IP that was close by?)\n* Time till 1st byte received for a given response\n* Time delay between send / acknowledge in the TCP stream\n* TLS handshake time\n\nBeyond network metrics, there are a number of other UX metrics to consider:\n\n* Time-to-interactivity (TTI): how long does it take for an app to become usable after the user has launched it? (This is an extremely important UX metric)\n* Total time a blank screen is shown during a user session\n* Total time loading animations / spinners are shown during a user session\n* Video quality (MOS)\n* Time to full load (for progressively-loaded apps)\n\n### What are some common mistakes mobile app developers make that lead to poor performance?\n\nWhen developing an application, we are often doing so on top-of-the-line desktop or laptop computers and devices, with fast corporate internet. The performance we experience during development may be so good that it masks issues experienced by the average set of users. Here are a few common mistakes (again, largely network-oriented) that developers make which can radically impact performance:\n\n* Using HTTP/2 or a custom TCP communication channel instead of sticking with the simplicity of HTTP v1 and optimizing traffic. (I.e., there aren't as many gains as you might expect from a more modern or complicated network stack)\n* Having dependent/blocking network requests that are executed serially; whenever possible, network requests should be executed in parallel.\n* Having too many network requests on a single view. This can be a consequence of a microservices / micro API architecture where to get the appropriate data, lots of requests are required. For mobile it is essential to make as few requests as possible (ideally only 1 per view), and therefore to sacrifice purity / composability of API response in order to aggregate all the data necessary to build a given view.\n* Misconfigured DNS causing devices to talk to servers that are unduly far away.\n* Unoptimized images, video, or audio being sent to regions or networks that can't handle the bandwidth.\n\n### How should we think about absolute vs relative performance measurements?\n\nIn general, it can often be more useful to track performance relative to a certain baseline, whether that is an accepted standard baseline, or just the first point at which you started measuring performance of your app. However, tracking relative performance can also be a challenge when testing across a range of devices or networks, because relative measures might not be comparing apples to apples. In these cases, looking at absolute values side-by-side can be quite useful as well.\n\n### Are there any absolute standard performance targets generally recognized as helpful?\n\nIt's true that each team defines UX for their own app. Acceptable TTI measures for an e-commerce app might differ by an order of magnitude or more from acceptable measures for AAA gaming titles. Still, there are some helpful rules of thumb based on HCI (Human-Computer Interaction) research:\n\n* Any delay over 500ms becomes a \"cognitive\" event, meaning the user is aware of time having passed.\n* Any delay over 3s becomes a \"reflective\" event, meaning the user has time to reflect on the fact of time having passed. They can become distracted or choose to go do something else.\n\nThese are not hard-and-fast truths that make sense in every case. And of course nobody really has a universal answer, but again, it's helpful to treat that 500ms number as a good target for any interaction we want to feel \"snappy\".\n\n(To follow up on some of this research, read up on the [human processor model](https://en.wikipedia.org/wiki/Human_processor_model) or [powers of 10 in UX](https://www.nngroup.com/articles/powers-of-10-time-scales-in-ux/))\n\n### How widely should we expect performance to vary across different devices?\n\nIn other words, when should we be really concerned about differences in performance between different devices or networks? Actually, it's fairly common to see differences of about 30% as quite common between devices. This level of difference doesn't usually indicate a severe performance issue, and can (with all appropriate caveats) be regarded as variance.\n\nTrue performance problems can cause differences of 10-100x the baseline measurements--just think how long you've waited for some app views to load when they are downloading too much content over a slow network!\n\n### How do you decide which devices to focus on for performance testing?\n\nThe answer here is simple if not practical: test on the devices that bring in the greatest revenue! Obviously this implies that you have some kind of understanding of your userbase: where are they located? What devices do they use? What is their typical network speed? And so on. If you don't have this information, try to start tracking it so you can cross-reference with whatever sales metrics are important for your product (items purchased, time spent in app, whatever).\n\nAt that point, if you can pick the top 5 devices that meet these criteria in a given region, you're well positioned to ensure a strong UX.\n\n### Conclusion\n\n\"Performance\" turns out to be quite a broad subcategory of UX, and of course, what we care about at the end of the day is UX, in a holistic way. The more elements of the UX we can begin to measure, the more we will be able to understand the impact of changes in our application. We'll even eventually get to the point where we've identified solid app-specific metric targets, and can fail builds that don't meet these targets, guaranteeing a minimum high level of UX quality for our users. Oh, and our users? They won't know any of this is happening, but they'll *love* you for it.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Mobile App Performance Testing","shortDesc":"What is mobile app performance? How do we know what to track and how to interpret what we find? In this part-FAQ, part-interview, we explore a host of topics that take us to the edge of thinking about mobile app performance as a significant component of mobile app UX.","liveAt":"2020-01-15 10:00","canonicalRef":"https://www.headspin.io/blog/mobile-app-performance-testing"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/102-mobile-app-performance-testing","slug":"102-mobile-app-performance-testing","path":"/editions/102-mobile-app-performance-testing","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0101.md","num":101,"rawMd":"\n\n> Appium Pro is normally all about... well, Appium! And other mobile testing related topics. However, in this post we're going to discuss an exciting development in AI in the world of Selenium, Appium's web-based forebear. Read on--I think you'll get something out of this even if you're focused purely on mobile testing.\n\nFor some time Appium has been experimenting with AI/ML approaches to augmenting mobile test automation. In addition to its [visual testing capabilities](https://appiumpro.com/editions/98), there is also a special plugin for [finding elements using ML models](https://appiumpro.com/editions/39) (even when [all you have is a screenshot](https://appiumpro.com/editions/92)). Part of what makes these features possible with Appium is the fact that it is possible to write plugins for Appium that integrate with these various other projects.\n\nI've often wondered how we can do the same thing with Selenium. Unfortunately, Selenium's architecture is not quite set up for third parties to write plugins that take advantage of behavior in the Selenium server itself. That doesn't stop us from writing client-side \"plugins\" that have access to the `driver` object, though!\n\n### AI-based element finding for Selenium\n\nHow do we create a client-side plugin for Selenium? Basically by putting together a library which takes an existing Selenium session (a `driver` object) and uses it for its own purposes. In our case, this special library will have access to the Test.ai classification model that already exists as part of the [Test.ai + Appium classifier plugin](https://github.com/testdotai/appium-classifier-plugin). This plugin was originally developed to give Appium users access to the classification model via the `-custom` locator strategy. The advantage of this approach was precisely that it was the Appium _server_ being augmented--all the work could be done in one language (Node.js) and made available to every client library with minimal modifications.\n\nIn the case of Selenium, the equivalent work would have needed to be done as an extension to _each_ client library. That was way too much work! So instead, we extended the capabilities of the existing Appium classifier plugin, so that it could also act as a classification _server_. This approach (very much akin to the client/server architecture of Selenium and Appium themselves) keeps the heavy lifting in one place, and allows very thin clients to be written in every language. The only downside is that you have to make sure to have the classifier server up and running.\n\n### The Classifier server\n\nIf you already have the `test-ai-classifier` package installed via NPM, no extra install steps are necessary. Otherwise, `npm install -g test-ai-classifier`. Then, running the server is quite simple:\n\n```\ntest-ai-classifier\n```\n\nWith no arguments, the server will start up on localhost, port 50051 (the default for gRPC-based services). Of course, you can always pass in `-h` and `-p` flags with custom host and port information (using `0.0.0.0` for host if it's important to listen on all interfaces).\n\n### The Classifier client\n\nOnce you've got the server running, you need to decide which client to use. There are four available:\n\n* [Java client](https://github.com/testdotai/classifier-client-java)\n* [Python client](https://github.com/testdotai/classifier-client-python)\n* [Node client](https://github.com/testdotai/classifier-client-node)\n* [Ruby client](https://github.com/testdotai/classifier-client-ruby)\n\nWe'll use the Java client for our purposes. To get it included in you Gradle-based Java project, the easiest thing to do is use [Jitpack](https://jitpack.io), and then to include a directive like the following, to get the client downloaded from GitHub:\n\n```gradle\ntestImplementation group: 'com.github.testdotai', name: 'classifier-client-java', version: '1.0.0'\n```\n\nThere are a few different ways to use the client, including the ability to pass image data to it directly, outside of the context of Appium, Selenium, or anything else. Either way, the first thing we need to do is instantiate the client:\n\n```java\nclassifier = new ClassifierClient(\"127.0.0.1\", 50051);\n```\n\nThe only parameters are the expected host and port values. Of most interest for us in terms of what we can call on `classifier` here is the method `findElementsMatchingLabel`, which takes two parameters: a `driver` object and a string representing the label for which we want to find matching elements. Have a look at this example:\n\n```java\nList els = classifier.findElementsMatchingLabel(driver, \"twitter\");\n```\n\nIn this case, we're looking for any elements that look like a Twitter logo. Notice that the return value of this method is exactly what you'd expect--a list of standard `WebElement` objects. You can click them, get their attributes, and anything else you'd be able to do with a regular element.\n\nHow does all this magic work? Well, the Classifier client runs a special XPath query that attempts to find any leaf node element, and then directs the browser to take a screenshot of each element, all on its own. From these screenshots, the client has all the image data it needs to send over to the Classifier server, which sends back information about the strength of any matches. The client can then map these results to the elements it found via XPath, filter out any which _don't_ match the requested label, and return the rest to you!\n\nWhat this does mean is that any browser driver you use will need to support the \"take element screenshot\" command. In my experimentation, only Chrome was reliable enough to not fail in weird ways when asked to take screenshots of so many elements. This API is relatively new, so I expect we'll see better reliability from Safari and Firefox (the only two other browsers I tried) soon enough. At any rate, take a look at the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition101_AI_For_Selenium.java) below, which demonstrates how we can load up a webpage, find an icon using only its semantic label, and then interact with it:\n\n```java\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.List;\n\nimport org.hamcrest.collection.IsCollectionWithSize;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.chrome.ChromeOptions;\nimport org.openqa.selenium.remote.RemoteWebDriver;\n\nimport ai.test.classifier_client.ClassifierClient;\n\npublic class Edition101_AI_For_Selenium {\n private RemoteWebDriver driver;\n private ClassifierClient classifier;\n\n @Before\n public void setUp() throws MalformedURLException {\n driver = new RemoteWebDriver(new URL(\"http://localhost:4444/wd/hub\"),\n new ChromeOptions());\n classifier = new ClassifierClient(\"127.0.0.1\", 50051);\n }\n\n @After\n public void tearDown() throws InterruptedException {\n if (driver != null) {\n driver.quit();\n }\n if (classifier != null) {\n classifier.shutdown();\n }\n }\n\n\n @Test\n public void testClassifierClient() throws Exception {\n // navigate to a webpage\n driver.get(\"https://test.ai\");\n\n // find the twitter icon\n List els = classifier.findElementsMatchingLabel(driver, \"twitter\");\n\n // make sure we have just one element which is a twitter icon, and click on it\n Assert.assertThat(els, IsCollectionWithSize.hasSize(1));\n els.get(0).click();\n\n // assert that we got to the appropriate twitter homepage\n Assert.assertEquals(driver.getCurrentUrl(), \"https://twitter.com/testdotai\");\n }\n}\n```\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"AI for Appium--and Selenium!","shortDesc":"Appium and Test.ai have been partnering on bringing AI to open source testing for some time. In this article we take a look at how the technology which powers the Appium element classification plugin can also be applied to Selenium, with reference to special new client libraries created for Java and other languages that can help bring a little bit of AI to your Selenium tests.","liveAt":"2020-01-08 10:00","canonicalRef":"https://www.headspin.io/blog/ai-for-appium-and-selenium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/101-ai-for-appium--and-selenium","slug":"101-ai-for-appium--and-selenium","path":"/editions/101-ai-for-appium--and-selenium","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0100.md","num":100,"rawMd":"\n\n> This post is the third in a 3-part series on visual testing with Appium. Visual testing is an important category of testing and can catch visual regressions as well as functional bugs. The inspiration for this series comes from a [free online video course on mobile visual testing](https://testautomationu.applitools.com/appium-visual-testing/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-100_visual-testing-with-appium-part-3&utm_content=uta&utm_term=cloud_grey) I produced for [Test Automation University](https://testautomationu.applitools.com/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-100_visual-testing-with-appium-part-3&utm_content=uta&utm_term=cloud_grey). Check it out! (And don't forget to read the [first](https://appiumpro.com/editions/98) and [second](https://appiumpro.com/editions/99) parts in this series first).\n\nIn Part 1 of this series, we put together our own homebrew visual testing solution. In Part 2, we discussed 4 limitations of that simple solution, and showed ways to ameliorate some of those difficulties by taking advantage of the Applitools visual testing SDK. We did not, however solve _all_ of the problems. There are a couple issues (which we listed out earlier) still remaining that we need to address:\n\n3. Applitools _does_ take care of some annoying ever-changing bits of the device screen. By default, for example, we will not run into any issues with the device time changing. But what about content inside our app that changes from test to test?\n4. The equivalent of \"full-page screenshots\"--so far, we haven't seen how to take screenshots and make comparisons using _all_ of the items in a list view.\n\n### Enabling full-page visual comparison\n\nLet's take #4 first, since it's relatively easy to address within the context of the Eyes SDK. It's actually just an SDK flag; to have Applitools automatically scroll any scrollable views and stitch together content to form one long screenshot, call the `setForceFullPageScreenshot` command with the value `true`:\n\n```java\neyes.setForceFullPageScreenshot(true);\n```\n\nThe reason this functionality is not enabled by default is that there is a certain amount of overhead involved in detecting scrollable views and then in actually scrolling them, so it should be turned on only when you know the comparison will be worth the extra time. Of course, you can also turn it on and off throughout the course of a test.\n\nLet's take a look at this functionality in action. The two versions of my test app that we've been using in examples so far both contain a home screen with a number of elements on it. What happens if I run a visual test comparing the home screen of V2 of the app with the baseline generated from V1 of the app? The code I'm running looks something like this:\n\n```java\n@Test\npublic void testHomeScreen() {\n eyes.open(driver, \"TheApp\", \"appium pro home screen scroll test\");\n\n // wait for an element that's on the home screen\n waitForElement(wait, LOGIN_SCREEN);\n\n // now we know the home screen is loaded, so do a visual check\n eyes.checkWindow(CHECK_HOME);\n eyes.close();\n}\n```\n\nThis is about as simple a test as we can imagine; it just waits for an element to appear on the home screen, and performs a visual check. When I run this test on Applitools, I get a visual differences warning. This is to be expected, because I have added some items to the home screen in V2 of the app. But have a look at the Applitools page where I go to have a look at the differences:\n\n

\n \"Visual\n

\n\nNotice that the screenshots are of very different heights. This is because, with V2 of the app, we scrolled the home view to make sure to catch all the different elements. Thus here we get a really good view of everything that changed. Without full-screen scroll, we might notice a change, but not the extent of it. And if we had a V3 of the app which added even more items, we wouldn't notice any differences at all without scrolling.\n\n### Ignoring unimportant visual differences\n\nThe final problem we want to solve in order to have ourselves a nice and robust visual testing pipeline is the problem of false positives. How can we make sure our visual testing logic isn't tripped up by visual differences that don't actually mean anything?\n\nFor example, in our test app there is a little \"echo screen\" that takes some user input and then displays it back on the screen. We could write a little functional-plus-visual test of this functionality with some code like that below:\n\n```java\n@Test\npublic void testEchoScreen() {\n String msg = \"hello\";\n eyes.open(driver, \"TheApp\", \"appium pro echo screen test\");\n\n wait.until(ExpectedConditions.presenceOfElementLocated(ECHO_SCREEN)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(MSG_BOX)).sendKeys(msg);\n driver.findElement(SAVE_BTN).click();\n WebElement savedMsg = wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.AccessibilityId(msg)));\n Assert.assertEquals(msg, savedMsg.getText());\n\n eyes.checkWindow(CHECK_ECHO);\n eyes.close();\n}\n```\n\nThis code is nothing new; it just finds and interacts with elements in very standard ways. The first time we run this test, there are no visual errors. In fact, we could run it again and again, and there would be no problem. But what if, to satisfy some external requirement, we needed to change the `msg` variable to contain a different string, like `\"world\"` instead of `\"hello\"`? The next time we run the test, we'll be greeted with visual differences that trigger a warning:\n\n

\n \"Unignored\n

\n\nThis is reasonable, but tedious. We didn't really change anything about the visual design of the app, and this is not a bug. So why waste our time looking at false positives like these? We can actually direct Applitools to _ignore_ regions of the screen that change frequently or only with content that is unimportant to visual testing. To do that, we simply choose the \"ignore region\" tool and draw boxes around the areas we don't care about tracking:\n\n

\n \"Ignore\n

\n\nOnce we save these regions and the baseline, we can update our code again (maybe making the `msg` variable contain the string `\"Appium\"`), and rerun the test. This time, we don't get any notice of visual difference, and if we log into the Applitools app, we can see that the check was marked automatically as \"passed\", even though we can see quite clearly that the strings differed between the two runs:\n\n

\n \"Ignore\n

\n\nThis is exactly what we want! Now we've decoupled the text used in the functional aspect of the test from the visual checks. If you want to see the code for both of the above sections in the context of a runnable project, you can [check it out on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition100_Visual_Testing_3.java).\n\n### Visual testing conclusion\n\nIn this 3-part series, we've seen how to use Appium's built-in image processing technology (powered by OpenCV) to build out some basic visual testing capabilities. We've also looked at the main problems that are important to solve in order to have a functional, maintainable, and robust visual testing framework. Thankfully, projects like [Applitools](https://applitools.com/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-100_visual-testing-with-appium-part-3&utm_content=uta&utm_term=cloud_grey) exist which provide some of these solutions in an off-the-shelf fashion through the use of language-specific SDKs. Of course, you could also go the route of finding solutions to the basic framework's limitations on your own or through a combination of other tools. Either way, I hope this series has encouraged you to get started with adding at least a few visual tests to your build, if you don't have any yet! And if you want a more in-depth version of these brief tutorials, don't forget to check out [my mobile visual testing course](https://testautomationu.applitools.com/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-100_visual-testing-with-appium-part-3&utm_content=uta&utm_term=cloud_grey) on Test Automation University.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Visual Testing With Appium, Part 3","shortDesc":"In Part 3 of this visual testing series, we tackle the remaining problems that can plague a mobile visual testsuite, including the challenge of full-scroll screenshots and what to do about screen regions that host changing content and can lead to false positive results.","liveAt":"2019-12-18 10:00","canonicalRef":"https://www.headspin.io/blog/visual-testing-with-appium-part-3"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/100-visual-testing-with-appium-part-3","slug":"100-visual-testing-with-appium-part-3","path":"/editions/100-visual-testing-with-appium-part-3","news":{"rawMd":"Believe it or not, you're reading Appium Pro edition #100! It's hard to believe it's been almost 2 years since I (and other awesome folks like Jonah) have been writing one of these articles every single week. When I set out on this mission, I imagined that I would run out of ideas for content a long time ago, but my list of future topics just keeps growing.\n\nTo mark this \"centarticle\", Appium Pro is going to take a holiday next week (December 25--happy Christmas to those of you who celebrate it!). If you want something to help meet your test automation content needs, I recommend having a read through [last year's holiday article](https://appiumpro.com/editions/49) and following its advice to script a special donation to a worthy cause.\n\nAlso, in 2020 I'll be looking for one (or a few) dedicated Appium Pro readers to help me take the site and newsletter beyond Java into other programming languages, beginning with Python. If you're an Appium expert who loves writing Python as well as writing words, give me a shout through the [contact form](https://appiumpro.com/contact) (or just reply to this e-mail). Please don't be shy--this is a paid gig.\n\nHave a great set of end-of-year rituals and holidays, and I'll be back in your inboxes in 2020!\n","num":100,"mdPath":"/vercel/path0/content/news/0100.md"},"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0099.md","num":99,"rawMd":"\n\n> This post is the second in a 3-part series on visual testing with Appium. Visual testing is an important category of testing and can catch visual regressions as well as functional bugs. The inspiration for this series comes from a [free online video course on mobile visual testing](https://testautomationu.applitools.com/appium-visual-testing/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-99_visual-testing-with-appium-part-2&utm_content=uta&utm_term=cloud_grey) I produced for [Test Automation University](https://testautomationu.applitools.com/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-99_visual-testing-with-appium-part-2&utm_content=uta&utm_term=cloud_grey). Check it out! (And don't forget to read the [first part](https://appiumpro.com/editions/98) in this series first).\n\nIn the [first part](https://appiumpro.com/editions/98) of this series, we looked at how to put together a simple but effective homebrew visual testing framework for mobile apps using Appium and Appium's built-in image processing capabilities (thanks, OpenCV!). Our approach did have a number of limitations, however:\n\n1. We had to manage our own baseline images by storing them in a directory on the machine running Appium. This is obviously not very scalable, since whole teams would need access to these images, we'd probably want to keep a revision history in case we want to roll back to a previous baseline, etc...\n2. We had to manage the logic for comparing screenshots to baselines, including deciding what to do when we detected differences. In the case of the logic we implemented, we simply threw an exception when differences were detected. But this would kill our functional test as well! It's nice for the functional test to continue and for visual differences to be merely *flagged* for later review rather than cause the whole test to fail.\n3. In our old system, if we detected a visual difference, an image would be generated for us that shows us where the differences might be. It would look something like this, which is an example of how things show up when encountering a failure:\n

\n \"Example\n

\n As you can see, it's not just the dislocated \"Login\" button which is highlighted. Various bits of the device UI have also changed. How long until there are *enough* changes in this part of the UI for a visual test to fail, when in reality nothing changed in our app? In general, we have to worry a lot about *false positives*, which is when we are alerted to a potential problem when none in fact exists. Other typical causes for false positives include, for example, dynamic text whose content is irrelevant to the visual check but which does visually change every time the test is run.\n4. The current approach doesn't at all handle scrolling content. What happens when we want to visually verify a screen which has a list or a scroll view? For this to work, we'd need to use Appium to incrementally scroll, take screenshots, and stitch them together into one long image for comparison. This is certainly possible, but it can be pretty finicky.\n\n### Making visual testing robust and maintainable\n\nThankfully, we can solve pretty much all of these challenges. My favorite way to handle them is with a service called [Applitools](https://applitools.com/?utm_source=influencer&utm_medium=appium&utm_campaign=edition-99_visual-testing-with-appium-part-2&utm_content=uta&utm_term=cloud_grey), which specializes in exactly this kind of visual testing. Let's see how I can eliminate a bunch of code from the last article and at the same time improve the quality and developer experience of our visual testing.\n\nFirst, I have to ensure that I have the Applitools Eyes SDK set up in my project:\n\n```gradle\ntestImplementation group: 'com.applitools', name: 'eyes-appium-java4', version: '4.2.1'\n```\n\nNow, I can adjust my `setUp` and `tearDown` methods to manage the state of the Eyes service, represented by a class field named `eyes`:\n\n```java\n@Before\npublic void setUp() throws Exception {\n // caps and driver instantiation\n\n // set up Eyes SDK\n eyes = new Eyes();\n eyes.setLogHandler(new StdoutLogHandler());\n eyes.setApiKey(System.getenv(\"APPLITOOLS_API_KEY\"));\n}\n\n@After\npublic void tearDown() {\n eyes.abortIfNotClosed();\n if (driver != null) {\n driver.quit();\n }\n}\n```\n\nHere I am merely telling Eyes to log its output to stdout so I can read it from the console, and also registering my API key (which I've stored as the value of an environment variable) so that the service knows to associate checks and screenshots with me and not someone else.\n\nNow, for each distinct visual test session, I call `eyes.open` and `eyes.close`, like so:\n\n```java\neyes.open(driver, \"TheApp\", \"appium pro basic design test\");\n// actual functional test logic and visual checks go here\neyes.close();\n```\n\nNotice that this is where we actually give the `eyes` object access to our `driver`, which is how it will be able to tell Appium to take screenshots at the appropriate times.\n\nIn the previous article, we had implemented a method for performing visual checks, called `doVisualCheck`. Now, we can completely delete that entire method. Instead, any time we want to perform a visual check based on the current screenshot, we call `eyes.checkWindow`. Like our own method, it takes a parameter which is the name of the check. So here's how our actual test method could look, mixing Appium's functional commands with Applitool's visual check commands:\n\n```java\n@Test\npublic void testAppDesign() {\n eyes.open(driver, \"TheApp\", \"appium pro basic design test\");\n\n WebDriverWait wait = new WebDriverWait(driver, 5);\n\n // wait for an element that's on the home screen\n WebElement loginScreen = waitForElement(wait, LOGIN_SCREEN);\n\n // now we know the home screen is loaded, so do a visual check\n eyes.checkWindow(CHECK_HOME);\n\n // nav to the login screen, and wait for an element that's on the login screen\n loginScreen.click();\n waitForElement(wait, USERNAME_FIELD);\n\n // perform our second visual check, this time of the login screen\n eyes.checkWindow(CHECK_LOGIN);\n eyes.close();\n}\n```\n\nThat's it!\n\n### Visual testing in action\n\nTo see how this new approach addresses some of the challenges, we'll need to actually run the test using two different versions of the app, to simulate two builds occurring at different points in time with app changes having occurred between them. I have uploaded two versions of The App here:\n\n```java\nString APP_V1 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-VR-v1.apk\";\nString APP_V2 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-VR-v2.apk\";\n```\n\nIf we first run the test with `APP_V1` as the value of the `app` capability, the test will pass. (And if you want to follow along, remember that the [full code for this article](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition099_Visual_Testing_2.java) is available). That's great! If I wanted, I could log into the Applitools dashboard and see that baselines have been captured from the run. But at this point, nothing interesting has happened because we've just set the baselines. So let's make things more interesting, and switch the `app` capability to `APP_V2`, and run the test again. This time, I'll get a failure, and the following output:\n\n```\nEdition099_Visual_Testing_2 > testAppDesign FAILED\n com.applitools.eyes.exceptions.DiffsFoundException: Test 'appium pro basic design test' of 'TheApp' detected differences! See details at: https://eyes.applitools.com/app/batches//\n at com.applitools.eyes.EyesBase.close(EyesBase.java:793)\n at com.applitools.eyes.EyesBase.close(EyesBase.java:735)\n at Edition099_Visual_Testing_2.testAppDesign(Edition099_Visual_Testing_2.java:81)\n```\n\nThat's actually a pretty handy error message. We even get a link so we can directly hop into the web app and start exploring the differences that were found. Once I log in, I can see the result of the most recent run:\n\n

\n \"Unresolved\n

\n\nIt's marked as \"unresolved\", and sure enough there are some apparent differences. I can click on the first difference, and I'm greeted with an interface that allows me to see the differences in more detail:\n\n

\n \"Detail\n

\n\nNotice first of all that there are no device status bars or clock times in these images. They don't matter to the app and so they aren't included at all. With this visual difference, it's pretty easy to tell what happened--the second version of the app added some list items! So this is not actually a regression, just a difference that's actually fine. I can let the tool know that this is all nominal by clicking the \"thumbs up\" button at the top right.\n\nThe next difference is a little more nasty:\n\n

\n \"Detail\n

\n\nWe can see here that a button has moved up. This doesn't look as good, so it's probably a visual regression. I can therefore click the \"thumbs down\" button, and the check as well as the overall visual test will be marked as failed. This is good because it accurately reflects the state of the app's quality. I can of course send this failure report immediately to my design or development team. Once they provide a new build, we can re-run the exact same test code, and verify that the new version is visually similar enough to the original baseline.\n\nThat's all there is to it! Now that I'm done walking through all the detected differences, I can simply save my choices, and Applitools will consider the new home screen image as the baseline for all future comparisons. Stay tuned for the next and final installment of this series, where we explore some advanced techniques that help solve even more of the problematic issues we described above about our original framework.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Visual Testing With Appium, Part 2","shortDesc":"In this second part of the series on visual testing, we think about some of the problems that exist with our home-grown visual testing solution, and explore how the Applitools Eyes SDK is well-suited to making our visual testing more robust and maintainable.","liveAt":"2019-12-11 10:00","canonicalRef":"https://www.headspin.io/blog/visual-testing-with-appium-part-2"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/99-visual-testing-with-appium-part-2","slug":"99-visual-testing-with-appium-part-2","path":"/editions/99-visual-testing-with-appium-part-2","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0098.md","num":98,"rawMd":"\n\n> This post is the first in a 3-part series on visual testing with Appium. Visual testing is an important category of testing and can catch visual regressions as well as functional bugs. The inspiration for this series comes from a [free online video course on mobile visual testing](https://testautomationu.applitools.com/appium-visual-testing/?utm_source=AppiumPro&utm_medium=newsletter&utm_campaign=Appium&utm_content=Appium&utm_term=Appium) I produced for [Test Automation University](https://testautomationu.applitools.com/?utm_source=AppiumPro&utm_medium=newsletter&utm_campaign=Appium&utm_content=Appium&utm_term=Appium). Check it out!\n\nWhy do we care about visual testing? Visual testing is a type of testing that overlaps with but is largely orthogonal to functional testing. Here's how I think about the differences:\n\n* **Functional testing** ensures that there are no bugs in the relationship between the user inputs and the code outputs of your app.\n* **Visual testing** ensures that there are no visual discrepancies between different test runs. Visual discrepancies could be a sign of a number of bugs, or a visual/aesthetic regression.\n\nVisual regressions can be just as bad news as functional regressions, especially when they surface bugs that a functional testsuite wouldn't catch. Here's a great example found in [@techgirl1908](https://twitter.com/techgirl1908)'s Twitter feed:\n\n

\n \"Example\n

\n\n## Steps for visual validation\n\nWhat do we need for visual validation? The basic principle is that, as we navigate through our app (in the course of a functional test, maybe), we take screenshots of each view we encounter. We then compare these screenshots with a previous screenshot of the same view, using some kind of image analysis software. If we encounter any notable differences, we may have found a visual regression, and at that point can either fail the test or simply flag the difference in some kind of database for a team to look at down the road.\n\n

\n \"Steps\n

\n\nAs pointed out in the diagram above, the images we use as a reference check are called \"baselines.\" Typically we gather baselines on the first visual test run, and then compare against the baselines on subsequent runs.\n\n## Visual comparison with Appium\n\nWe already know how to get screenshots with Appium, so that part is taken care of. But how do we compare two screenshots to check for visual differences? As a human being, this is pretty easy (unless you're playing one of those difficult spot-the-difference games), but how on earth can we teach a computer to do this?\n\nThe short answer is that there are lots of smart people out there who have come up with clever algorithms for detecting visual differences. A naive approach to visual difference (checking identity of each pixel across the two images) would be way too brittle, since even compression algorithms might result in almost unnoticeable differences. But luckily for us in the open source world, we have something called [OpenCV](https://opencv.org), which is a collection of image processing tools we can use without having to really understand in depth how they are implemented.\n\n### Additional system setup\n\nAppium has integrated support for OpenCV, though it does not come turned on by default, since building OpenCV and its Node.js bindings takes a lot of time and must take place on individual platforms. The easiest way to get things set up for use with Appium is to simply run `npm install -g opencv4nodejs`. This will attempt to globally install the Node bindings, which will also take care of downloading and building OpenCV on your system. If that doesn't work, you can install OpenCV using Homebrew and then install the Node bindings with the `OPENCV4NODEJS_DISABLE_AUTOBUILD=1` env flag in front, to tell it to use the system-installed binaries.\n\nOnce you've got the opencv4nodejs package installed, you also need to make sure it's accessible to Appium when it runs. One way of doing this is to run the `npm install` command *without* the `-g` flag, inside the Appium directory itself. Another solution is to ensure that your global `node_modules` folder is listed in an environment variable called `NODE_PATH`.\n\n### Image comparison commands\n\nOK, we've got all this OpenCV business set up, so how do we actually use it from our Appium test code? With this handy command:\n\n```java\nSimilarityMatchingResult res = driver.getImagesSimilarity(img1, img2, opts);\n```\n\nWhen you call this command with the appropriate image byte arrays (here, img1 would be our baseline, and img2 would be the screenshot we wish to validate), and with the appropriate options object (it needs to be an instance of `SimilarityMatchingOptions`), what you get back is a `SimilarityMatchingResult` object.\n\nThe all-important method on the result object is `getScore`, which will tell us how similar the two images we sent in were, in terms of a number between 0 and 1. To turn this into actual validation, we simply check whether the score is above a certain threshold (which should be experimentally chosen based on your app). If so, we consider there to be no significant differences. If not, we can either throw an exception to fail the test or flag the difference in some other way.\n\n```java\nif (res.getScore() < MATCH_THRESHOLD) {\n throw new Exception(\"Visual check failed!\");\n}\n```\n\nWhat's a good match threshold? I've been pretty successful with a value of 0.99, but your mileage may vary. Why do we need a value lower than 1.0? Because on any device screen there will be elements unrelated to the visual design of the app which are constantly changing--the time on the clock, if nothing else!\n\n### Visualizing the differences\n\nKnowing that two images differ in some way is great, but that's not actionable information for a developer. What we want is the two images side-by-side, with any differences highlighted to make clear what the potential issues are. Thankfully we can direct the `getImagesSimilarity` command to give us one of these human-readable images by setting the option object appropriately:\n\n```java\nSimilarityMatchingOptions opts = new SimilarityMatchingOptions();\nopts.withEnabledVisualization();\n```\n\nIf we've done this, then our result object will have another method on it:\n\n```java\nres.storeVisualization(fileObj);\n```\n\nThis method allows us to save the visualized differences image into a `File` object.\n\n### Hacking together a visual validation framework\n\nAll of what we've seen so far is enough to check the similarity between screenshots, but this is not yet a visual validation *system*. For that, we need baseline screenshot management. The simplest way we can implement this is as follows:\n\n1. Specify a directory on the system to contain baseline images.\n2. Make sure that each visual check we do has a unique name, so multiple baselines can exist for different views.\n3. Whenever we trigger a visual check, first look to see whether a baseline exists. If it does not, then simply save the screenshot as the baseline. If it does, then compare the current screenshot with the baseline.\n\nIn code, we might construct a `doVisualCheck` method that implements this strategy (assuming all the static class variables have been set correctly):\n\n```java\nprivate void doVisualCheck(String checkName) throws Exception {\n String baselineFilename = VALIDATION_PATH + \"/\" + BASELINE + checkName + \".png\";\n File baselineImg = new File(baselineFilename);\n\n // If no baseline image exists for this check, we should create a baseline image\n if (!baselineImg.exists()) {\n System.out.println(String.format(\"No baseline found for '%s' check; capturing baseline instead of checking\", checkName));\n File newBaseline = driver.getScreenshotAs(OutputType.FILE);\n FileUtils.copyFile(newBaseline, new File(baselineFilename));\n return;\n }\n\n // Otherwise, if we found a baseline, get the image similarity from Appium. In getting the similarity,\n // we also turn on visualization so we can see what went wrong if something did.\n SimilarityMatchingOptions opts = new SimilarityMatchingOptions();\n opts.withEnabledVisualization();\n SimilarityMatchingResult res = driver.getImagesSimilarity(baselineImg, driver.getScreenshotAs(OutputType.FILE), opts);\n\n // If the similarity is not high enough, consider the check to have failed\n if (res.getScore() < MATCH_THRESHOLD) {\n File failViz = new File(VALIDATION_PATH + \"/FAIL_\" + checkName + \".png\");\n res.storeVisualization(failViz);\n throw new Exception(\n String.format(\"Visual check of '%s' failed; similarity match was only %f, and below the threshold of %f. Visualization written to %s.\",\n checkName, res.getScore(), MATCH_THRESHOLD, failViz.getAbsolutePath()));\n }\n\n // Otherwise, it passed!\n System.out.println(String.format(\"Visual check of '%s' passed; similarity match was %f\",\n checkName, res.getScore()));\n}\n```\n\nNow, in our functional test cases, we can mix in visual checks however we like. Here's an example of a test that navigates and performs visual checks on a few views in the course of a basic functional verification:\n\n```java\n@Test\npublic void testAppDesign() throws Exception {\n WebDriverWait wait = new WebDriverWait(driver, 5);\n\n // wait for an element that's on the home screen\n WebElement loginScreen = waitForElement(wait, LOGIN_SCREEN);\n\n // now we know the home screen is loaded, so do a visual check\n doVisualCheck(CHECK_HOME);\n\n // nav to the login screen, and wait for an element that's on the login screen\n loginScreen.click();\n waitForElement(wait, USERNAME_FIELD);\n\n // perform our second visual check, this time of the login screen\n doVisualCheck(CHECK_LOGIN);\n}\n```\n\nThis is what we want! Of course, there are a lot of rough edges here, which you would run into if you tried to build an entire visual testsuite on top of this mini \"framework\" we've developed. So don't forget to check out episode 2 in this little series on visual testing with Appium. Oh, and if you want to see the complete code example in the context of a working project, you can always [find it on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition098_Visual_Testing_1.java).\n\nAs it stands, the tests will pass, but that's because they're running against just one version of an app. To see the real power of this approach in action, you will of course need to run the test against multiple versions of the app, where the second version actually differs visually from the first!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Visual Testing With Appium, Part 1","shortDesc":"You can use Appium for more than functional testing. Another very important category of testing is visual testing, wherein visual elements of the app are compared across runs to ensure that there are no visual discrepancies, which could be a sign of visual or even functional bugs. This is the first in a 3-part series showing how to use open source as well as industry tools to be successful in visual testing.","liveAt":"2019-12-04 10:00","canonicalRef":"https://www.headspin.io/blog/visual-testing-with-appium-part-1"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/98-visual-testing-with-appium-part-1","slug":"98-visual-testing-with-appium-part-1","path":"/editions/98-visual-testing-with-appium-part-1","news":{"rawMd":"In other news, I'll be giving a webinar next week with HeadSpin on [some of the lesser-known Appium drivers, and Appium 2.0](https://info.headspin.io/webinar/automate-everything). I'd love for you to join me; it'll be a lot of fun!\n","num":98,"mdPath":"/vercel/path0/content/news/0098.md"},"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0097.md","num":97,"rawMd":"\n\nWhile Appium is most often used for functional testing (does the functionality of my app behave?), it can also be used to drive other kinds of testing. Performance is a big and important area of focus for testers, and one key performance metric is app launch time. From a user's perspective, how long does it take from the time they tap the app icon to the time they can interact with it? App launch time can be a big component of user frustration. I know for myself that if an app takes too long to load, I often switch to something else and don't even bother using it unless I absolutely have to.\n\n### Using Appium To Capture Launch Time\n\nSo, how can we check this? The main trick is to use one little-known capability:\n\n```java\ncapabilities.setCapability(\"autoLaunch\", false);\n```\n\nSetting `autoLaunch` to false tells the Appium server to start an automation session, but *not* to launch your app. This is because we want to launch the app in a controlled and time-able manner later on. It's also important to remember that because Appium won't be launching the app for us, we're responsible for ensuring all app permissions have been granted, the app is in the correct reset state (based on whether we are timing first launch or subsequent launch), etc...\n\nOnce we have a session started, but no app launched, we can use the `startActivity` command to direct our app to launch, without any of the extra steps that Appium often takes during launch, so we get a relatively pure measure of app launch.\n\n```java\nActivity act = new Activity(APP_PKG, APP_ACT);\nact.setAppWaitActivity(APP_WAIT);\ndriver.startActivity(act);\n```\n\nWe might also care about not just how long the app takes to launch, but how long until an element is usable, so we can throw in a wait-for-element:\n\n```java\nwait.until(ExpectedConditions.presenceOfElementLocated(loginScreen));\n```\n\nBut this is just making the app launch; how do we actually measure the time it took? One technique would be to simply create timestamps in our client code before and after this logic, and get the difference between them. However, this would also count the time taken by the request to and response from the Appium server, which could add up to a lot depending on the network latency in between the client and server.\n\nIdeally, we'd look for a number on the *server* itself, and thankfully this is something we can do thanks to Appium's [Events API](https://appiumpro.com/editions/94). Using the Events API, we can determine when exactly the `startActivity` command started and ended on the server. We can also find the difference between the end of the `findElement` command and the beginning of the `startActivity` command, which gives us a (not quite as accurate) measure of the time to interactivity of the app. (The reason it's not as accurate is that we do potentially have some time lost due to latency of the `findElement` command itself, which we could work around using [Execute Driver Script](https://appiumpro.com/editions/85)).\n\nHere's how we use the Events API to accomplish this:\n\n```java\nServerEvents evts = driver.getEvents();\n\nList cmds = evts.getCommands();\nOptional startActCmd = cmds.stream()\n .filter((cmd) -> cmd.getName().equals(\"startActivity\"))\n .findFirst();\nOptional findCmd = cmds.stream()\n .filter((cmd) -> cmd.getName().equals(\"findElement\"))\n .findFirst();\n\nif (!startActCmd.isPresent() || !findCmd.isPresent()) {\n throw new Exception(\"Could not determine start or end time of app launch\");\n}\n\nlong launchMs = startActCmd.get().endTimestamp - startActCmd.get().startTimestamp;\nlong interactMs = findCmd.get().endTimestamp - startActCmd.get().startTimestamp;\n\nSystem.out.println(\"The app took total <\" + (launchMs / 1000.0) + \"s to launch \" +\n \"and total <\" + (interactMs / 1000.0) + \"s to become interactable\");\n```\n\nEssentially, we get the events from the server, and pull out the `CommandEvent`s for the commands we care about. Then we are free to compare their timestamps as necessary.\n\n### Comparing Performance Across Devices\n\nLooking at app launch on emulators or the device you have on hand locally is all well and good, but it's also important to get a good idea of the metric across many different devices, as well as over time across builds. The basic technique we used above of simply printing the metric to the screen is obviously not a scalable way to track performance. In a real test environment, you'd want to log that metric to a database of some kind, for example CloudWatch, DataDog, Splunk, etc... Then you'd be able to see it in a nice dashboard and set alerts for performance degradations with new builds.\n\nBut let's talk a little bit about running on multiple devices. Since I don't have access to a bunch of devices, I put together an example of how to run this app launch time test on [AWS Device Farm](http://bit.ly/device-farm). The way Device Farm works is that you set up a test \"run\", uploading your app as well as a zipfile of your test project. To facilitate building the zipfile of my Java test with all its dependencies, I added these handy scripts to my [build.gradle](https://github.com/cloudgrey-io/appiumpro/blob/master/java/build.gradle) file:\n\n```gradle\n// for getting jar for aws\n// source: https://stackoverflow.com/a/52398377\ntask marshallClasspathDeps(type: Copy) {\n from sourceSets.test.runtimeClasspath\n // if you need this from the dependencies in the build.gradle then it should be :\n // from sourceSets.main.runtimeClasspath\n include '**/*.class'\n include '**/*.jar'\n exclude { FileTreeElement details ->\n details.file.name.contains('Edition') || details.file.name.contains('kotlin')\n }\n into 'build/libs/dependency-jars'\n}\n\n//packaging tests task which is generated from sample using gradle init\ntask packageTests(type: Jar) {\n from (sourceSets.test.output) {\n include '**/*DeviceFarm*'\n }\n classifier = 'tests'\n includeEmptyDirs = false\n}\n\n// create zip archive\n// source: https://stackoverflow.com/a/36883221/8016330\ntask zip(type: Zip) {\n dependsOn packageTests, marshallClasspathDeps\n from 'build/libs/'\n include '*'\n include '*/*' //to include contents of a folder present inside dependency-jars directory\n archiveName 'zip-with-dependencies.zip'\n}\n```\n\nWith these scripts, whenever I need to generate a bundle for Device Farm, I can just run `./gradlew zip` (after making sure the old files are deleted). The only change I had to make to my test code itself was to make sure to get the path to my uploaded app via an environment variable set by DeviceFarm:\n\n```java\nString app = System.getenv(\"DEVICEFARM_APP_PATH\");\n```\n\nWhen you create a test run on AWS, you walk through a wizard that asks you for a bunch of information, including the app and test bundle. At some point, you'll also have the opportunity to tweak the test configuration, which is in YAML format:\n\n

\n \"Test\n

\n\nFor this example, I needed bleeding-edge Appium features that are only available in a beta version, so I put together a [custom Device Farm config](https://github.com/cloudgrey-io/appiumpro/blob/master/java/device-farm.yml) that I pasted into the config section and saved for reuse. Note that I wouldn't recommend doing this on your own, but instead use the default config and one of the tested and supported versions of Appium.\n\nOnce I've got the run all ready to go, I can choose which devices I want to run on. For this article, I chose to run on 5 devices:\n\n

\n \"Choose\n

\n\nDevice Farm's model is pretty different from other services I've used before. Rather than running the Appium client locally and talking to a remote server, the test bundle is uploaded to AWS via this wizard, and then the tests are scheduled and executed completely remotely. So at this point I sat back and waited until the tests were complete. When they were, I was able to see the results for all devices (they passed--eventually, anyway!)\n\n

\n \"Test\n

\n\nSince I just printed the launch time metrics to stdout, I had to dig through the test output to find them. When I did, these were the results:\n\n|Device|Time to launch|Time to interactivity|\n|------|--------------|---------------------|\n|Google Pixel|2.554s|3.263s|\n|Google Pixel 2|2.064s|2.291s|\n|Samsung Galaxy S6 (T-Mobile)|2.392s|5.201s|\n|Samsung Galaxy S6 (Verizon)|2.392s|4.468s|\n|Samsung Galaxy S9+|2.404s|2.647s|\n\nThat's it! I definitely encourage all of you to start tracking app launch time in your builds, as even a few seconds can make a big difference in user satisfaction. Want to check out the full source code for this article's test? As always, you can [find it on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition097_Tracking_App_Launch_DeviceFarm.java).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Capturing App Launch Metrics On Android","shortDesc":"App launch time is a significant performance metric, since it defines a user's first experience of your app. Luckily, with a bit of hackery, this is easy to determine using Appium. Combined with the use of a device cloud service (in this article we explore AWS Device Farm), it's within the reach of most test teams to track app launch performance over time.","liveAt":"2019-11-27 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-app-launch-metrics-on-android"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/97-capturing-app-launch-metrics-on-android","slug":"97-capturing-app-launch-metrics-on-android","path":"/editions/97-capturing-app-launch-metrics-on-android","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0096.md","num":96,"rawMd":"\n\nIf you've ever used Selenium, and if you're currently using Appium for web testing, you might have wondered if Appium supports interacting with [cookies](https://en.wikipedia.org/wiki/HTTP_cookie) stored in the browser. It does! And to explore how and why we might want to work with cookies, we're going to travel back in time to around 2006, when I was working as a freelance web developer. I had a side project called [Expenseus](http://expenseus.com), a useful little app designed to help keep track of shared expenses among a group of friends. Here's what it still looks like today:\n\n

\n \"Expenseus\"\n

\n\nDon't laugh! It wasn't pretty then and it certainly hasn't aged well, but it is still running after 13 years without any updates (please do not try and hack it). Above is an image of the login page. To build this app, for some reason I decided to write a more or less complete clone of Ruby on Rails in PHP (don't ask me why--I loved PHP that much, I guess!) As part of this whole framework, I built a cookie-based login system. The way this worked was that, on a visit to the site, the app would generate a unique session id for a visitor, and set that id in the `PHPSESSID` cookie. From then on, requests to the app would contain this unique id in the header. The server could then associate various bits of state with the user across requests, for example, whether a user is logged in, or whether the user just took an action which should result in a message being shown on the next page.\n\nThis is a very common pattern for cookie use. Even though cookies can store arbitrary data, they're usually just used to store a session token or id, and that is associated with state kept on the server in something like Memcache, or a database.\n\n### Using Cookies for Testing\n\nWhat does any of this have to do with *testing* websites? Well, we can leverage the use of cookies to save ourselves the task of setting up state that is already represented by a cookie. For example, if I have logged into my app, and I have access to the session id, I can take that session id, set it as a cookie in some other browser, and, voila, I'm logged in in the *other* browser now, too! As we all know, UI tests are slow, so anything we can do to avoid needlessly typing in fields and clicking buttons is worth a shot. So here's what we're going to do: build a test suite where the login steps are actually taken only *once*, and after that, cookies are used to log in more or less instantly for the other test cases. The logic goes like this:\n\n1. Navigate to the app's URL\n2. If we don't have a login cookie stored in our test class:\n 1. Navigate to the login page and log in by filling out fields and clicking buttons\n 2. Retrieve the session id from the login cookie and save it on our test class\n 3. Go to main step 4\n3. If we have a login cookie stored from a previous test:\n 1. Delete the current cookie\n 2. Set a new login cookie with the saved cookie information\n4. Proceed with the rest of the test, in a logged-in state.\n\n### The Cookie API\n\nIn the flow above, we need to interact with the cookies in three ways: getting cookie data, deleting a cookie, and adding a new cookie. The Appium / Selenium API has methods for all of these. Here's how we would get a cookie with the name `PHPSESSID`:\n\n```java\nCookie loginCookie = driver.manage().getCookieNamed(\"PHPSESSID\");\n```\n\nThe `Cookie` class gives us methods that allow us to retrieve the name, value, expiry, domain, and other metadata of the cookie. Here's how we would delete a cookie with the same name:\n\n```java\ndriver.manage().deleteCookieNamed(\"PHPSESSID\");\n```\n\nAnd finally, here's how we set a new cookie, assuming we have a `Cookie` instance called `loginCookie` (maybe the same object we got back as the result of the call above?) all ready to go:\n\n```java\ndriver.manage().addCookie(loginCookie);\n```\n\nPutting it all together, we can define a test helper method that will log a user in, either by automating the fields and buttons, or by setting a cookie if one exists in the static `loginCookie` class field:\n\n```java\nprivate void loginHelper() {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n driver.get(\"http://expenseus.com/user/login\");\n WebElement username = wait.until(ExpectedConditions.presenceOfElementLocated(\n By.xpath(\"//input[@name='user[email]']\")));\n if (loginCookie != null) {\n System.out.println(\"Using cookie\");\n driver.manage().deleteCookieNamed(COOKIE_NAME);\n driver.manage().addCookie(loginCookie);\n driver.get(\"http://expenseus.com\");\n } else {\n System.out.println(\"No cookie, logging in via form\");\n username.sendKeys(USERNAME);\n driver.findElement(By.xpath(\"//input[@name='user[password]']\")).sendKeys(PASSWORD);\n driver.findElement(By.xpath(\"//input[@value='Log in']\")).click();\n loginCookie = driver.manage().getCookieNamed(COOKIE_NAME);\n System.out.println(loginCookie.getName());\n System.out.println(loginCookie.getValue());\n }\n wait.until(ExpectedConditions.urlContains(\"dashboard\"));\n}\n```\n\n(In the code above, I've assumed some things about my app, for example the login url, the relevant locators, or the fact that the site takes me to the dashboard upon login.)\n\nThat's basically it! There are probably lots of other interesting ways to use cookies with automation (please let me know what you've come up with!), but it will of course depend partially on how your app is set up and how it uses cookies. Even the trick described here may not work in all cases, if for example the app's backend server ties login sessions not just to a cookie-based session id but also to a request IP.\n\nYou can check out a [full working example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition096_Cookies.java) of the technique above, where a test is run first on Safari on iOS, and then on Chrome on Android, where the Chrome test just uses the cookie retrieved from the Safari test, rather than having to go through all the login steps itself. It feels pretty magical to see it in action, until you remember that HTTP is totally stateless, so there's really no difference between two requests from one browser or two requests from two different browsers!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Working With Cookies","shortDesc":"When running tests of mobile websites with Appium, it can sometimes be advantageous to work with cookies that have been set in the browser by the application server, using the built-in cookie management commands.","liveAt":"2019-11-20 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-cookies"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/96-working-with-cookies","slug":"96-working-with-cookies","path":"/editions/96-working-with-cookies","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0095.md","num":95,"rawMd":"\n\nOne of the well-known limitations of finding elements on Android is that the Android UI renderer doesn't actually build and render elements until they're about to appear on the screen. What this means is that, if we are looking for an element which is far down in a list, and not yet visible on the screen, the element just does not \"exist\". You can prove this to yourself by generating a source snapshot of your app while there are lots of list elements present below the edge of the screen. They won't be there! Let's take as an example the ApiDemos app; let's say we're on the screen below, and want to tap on the \"Layouts\" item, which is not on the first screen's worth of items (you can see the first screen on the left, and how we had to scroll to get to \"Layouts\" on the right):\n\n

\n \"Need\n

\n\nHow do we handle a situation like this with Appium? Well, what I usually say is to \"think like a user\"--how would a user find this item? By swiping to scroll the list, of course, and using their eyes to periodically check for the appropriate content. We could, therefore, implement a scroll-and-find helper method with built-in retry loops. If we're using the Espresso driver, however, there's a better way: the 'Android Data Matcher' strategy!\n\nThis strategy takes advantage of the fact that Espresso runs in the app context, not on top of the Accessibility layer, so it has access to the data which backs a list, not just the elements which are displayed on the screen. (In Android, ListViews are a subtype of what's called an AdapterView, which connects the list datasource). We can use a part of the Espresso API to essentially target this *data*, and have Espresso automatically scroll to the element which represents that data for us.\n\nAs you can see in Appium's [Data Matcher Guide](http://appium.io/docs/en/writing-running-appium/android/espresso-datamatcher-selector/), if we were using Espresso directly (not through Appium), we might accomplish this feat with some code that looks like the following:\n\n```java\nonData(hasEntry(\"title\", \"textClock\")\n .inAdapterView(withId(\"android:id/list))\n .perform(click());\n```\n\nThis Espresso code waits for some \"data\" according to the parameters we've given the `onData` command, namely that we want an entry which has a specific title, within the list that has a certain Android ID. The important bit here is the `hasEntry` function, which builds a [Hamcrest Matcher](http://hamcrest.org/JavaHamcrest/javadoc/1.3/org/hamcrest/Matchers.html). The matcher is used by Espresso to help find the desired data and turn it into a view we can interact with.\n\nThere are all kinds of matchers, which you can explore and use with Espresso, and now with Appium. So how do we do this with Appium? Basically, we need to use the `-android datamatcher` locator strategy, which in Java is accessed as `MobileBy.androidDataMatcher`. And for the selector, rather than a simple string, we construct a JSON object containing the Matcher information, stringify it, and pass it as the selector. It's a bit sneaky (or, how I like to think of it, a clever way of working within the W3C WebDriver protocol definitions for locator strategy selectors). Here's how we'd construct the attempt to find the \"Layouts\" item we saw in the screenshot above:\n\n```java\n// first, find the AdapterView (not necessary if there's only one)\nWebElement list = wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.className(\"android.widget.ListView\")\n));\n\n// Construct the Hamcrest Matcher spec\nString selector = new Json().toJson(ImmutableMap.of(\n \"name\", \"hasEntry\",\n \"args\", ImmutableList.of(\"title\", \"Layouts\")\n));\n\n// find and click on the element using the androidDataMatcher selector\nlist.findElement(MobileBy.androidDataMatcher(selector)).click();\n```\n\nAs you can see, we have to do a bit of work to codify the Hamcrest Matcher we want to use, by settings its `name` and `args`. What should the values here be? Basically, whatever they would be in the raw Espresso case (the name of the matcher is `hasEntry`, and its arguments are `title` and `Layouts`, because we're trying to find an entry whose title is 'Layouts').\n\nThat's basically it! When you see this work, it's pretty awesome--the screen will jump very quickly to the expected item, without having to do a bunch of manual scrolling. Below is a full example, with two tests; one attempting to find the 'Layouts' item using XPath alone, which fails (because the element is not on the screen), and a second test which succeeds because it uses this locator strategy. This strategy is an especially good replacement for XPath when you only have partial information about the element you want to find.\n\n```java\npublic class Edition095_Android_Datamatcher {\n private AndroidDriver driver;\n private String APP = \"https://github.com/appium/android-apidemos/releases/download/v3.1.0/ApiDemos-debug.apk\";\n WebDriverWait wait;\n\n @Before\n public void setUp() throws Exception {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"Espresso\");\n capabilities.setCapability(\"app\", APP);\n\n driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testFindHiddenListItemNormally() {\n wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.AccessibilityId(\"Views\")\n )).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.xpath(\"//*[@content-desc='Layouts']\")\n )).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.AccessibilityId(\"Baseline\")\n ));\n }\n\n @Test\n public void testFindHiddenListItemWithDatamatcher() {\n wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.AccessibilityId(\"Views\")\n )).click();\n WebElement list = wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.className(\"android.widget.ListView\")\n ));\n String selector = new Json().toJson(ImmutableMap.of(\n \"name\", \"hasEntry\",\n \"args\", ImmutableList.of(\"title\", \"Layouts\")\n ));\n list.findElement(MobileBy.androidDataMatcher(selector)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(\n MobileBy.AccessibilityId(\"Baseline\")\n ));\n }\n}\n```\n\n(Of course, you can always view the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition095_Android_Datamatcher.java) on GitHub!)\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"The 'Android Data Matcher' Locator Strategy","shortDesc":"Sometimes, it is impossible or inconvenient to find elements that have not yet been rendered by the Android UI subsystems. Using the Espresso driver, we have access to a special locator strategy that enables us to access elements which have data bound to them, even if the elements themselves are not yet visible.","liveAt":"2019-11-13 10:00","canonicalRef":"https://www.headspin.io/blog/the-android-data-matcher-locator-strategy"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/95-the-android-data-matcher-locator-strategy","slug":"95-the-android-data-matcher-locator-strategy","path":"/editions/95-the-android-data-matcher-locator-strategy","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0094.md","num":94,"rawMd":"\n\n

\n \"Event\n

\n\nEver wondered what goes on under the hood while Appium is starting up, say, an iOS session using the XCUITest driver? Sometimes this is a process that takes several minutes, and sometimes it only takes a handful of seconds. Having a look at the Appium logs can clear up this confusion (maybe you notice that the first time you run an iOS test with a new version of Appium, it spent time rebuilding WebDriverAgent--that makes sense).\n\nJust looking at the logs doesn't necessarily tell you how *long* Appium spent within each zone of its startup routine, though. You could turn on timestamps in the logs (using `--log-timestamp`), but then you have to do a bunch of mental gymnastics to develop a picture of the startup flow over time.\n\n### Getting event data\n\nFor this reason, the Appium team developed something called the \"Events API\" (or sometimes the \"Event Timings API\"). Basically, the timestamps of various points of interest in the lifecycle of a session are logged automatically as \"events\", which can be retrieved at any point during the test.\n\nPreviously, it was possible to retrieve events using the `getSessionDetails` command:\n\n```java\nMap details = driver.getSessionDetails();\nMap events = details.get('events');\n```\n\nFor this to work, you also had to use the `eventTimings` capability, and set it to `true`, in order for Appium to decorate the session details response with the appropriate timing information. However, in the W3C protocol there is no route that corresponds to `getSessionDetails`, and so the Appium team has added its own route, which is supported in the Java client as the `getEvents` command:\n\n```java\nServerEvents events = driver.getEvents();\n```\n\n### Understanding event data\n\nWhat you get back from this call is a `ServerEvents` object encapsulating the various types of data returned with this API. Let's take a look at the (truncated) raw API response for an example call, so you can see its shape:\n\n```json\n{\n \"commands\": [\n {\n \"cmd\": \"findElement\",\n \"startTime\": 1573001594602,\n \"endTime\": 1573001594810\n },\n {\n \"cmd\": \"getLogEvents\",\n \"startTime\": 1573001599636,\n \"endTime\": 1573001599636\n }\n ],\n \"xcodeDetailsRetrieved\": [\n 1573001580191\n ],\n \"appConfigured\": [\n 1573001586967\n ],\n \"resetStarted\": [\n 1573001586969\n ]\n}\n```\n\nThere are two main sections here, one labeled `commands`, and then a bunch of other names consisting of timestamps. All commands sent to the Appium server are considered events, and their start and end times are tracked. In addition, the hand-picked named events are included, for example `xcodeDetailsRetrieved` or `appConfigured` above. (In all cases, the numeric values are timestamps in milliseconds.)\n\nThe `ServerEvents` object for this reason has a `commands` field and an `events` field, each constituting a list of their own type (`CommandEvent` and `ServerEvent`) respectively, and containing the event name and timestamp(s).\n\n### Custom events\n\nIn addition to the hand-picked events automatically included by the Appium team, you can also tell the Appium server to include your *own* events within the event log, so that you can have a record of when actions within the world of your test logic took place with respect to the built-in Appium commands. To do this, you need to log the event:\n\n```java\nCustomEvent evt = new CustomEvent();\nevt.setVendorName(\"prefix\");\nevt.setEventName(\"eventName\");\ndriver.logEvent(evt);\n```\n\nIn the example above, we are logging an 'eventName' event with the vendor name 'prefix'. Why do we need this vendor prefix? To ensure that custom events can never conflict with built-in ones, or with events logged by another library. These custom events will show up when you call `getEvents` later on! Here are some that I triggered in a test of my application:\n\n```\n ...\n \"theapp:onLoginScreen\": [\n 1573001595285\n ],\n \"theapp:testEnd\": [\n 1573001599630\n ]\n ...\n```\n\nIf you want, you can use the Java client to write out the raw data in JSON format somewhere on disk for use with other applications, too:\n\n```js\n// assume we have ServerEvents object 'events'\nevents.save(new File('/path/to/events.json').toPath());\n```\n\nWith all of this raw data, you could build interesting visualizations of the time Appium spent in between each of these events. But you don't really need to, because the Appium team already built a tool to do this for you! It's called the [Event Parser](https://github.com/appium/appium-event-parser), and it's a CLI tool you install via NPM:\n\n```bash\nnpm install -g appium-event-parser\n```\n\nYou can use the Event Parser to generate a timeline of all the events that are described in a JSON file, for example. (Did you notice the `save` command above? It gets you exactly what you need to make this happen).\n\n```bash\nappium-event-parser -t -i /path/to/events.json\n```\n\n(The `-i` flag is followed by the path to the data file, and `-t` tells the program you want a timeline printed out. You can also pass an `-l` flag followed by a number to change the number of lines the timeline takes up). Once you run this command, you'll be greeted by a nice little timeline:\n\n

\n \"Event\n

\n\nYou can also check out a [full sample test](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition094_Events_API.java) using these features! Let me know if you come up with any interesting use cases for this API; I'm sure there's some great possibilities out there.\n\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Using the Appium Events API","shortDesc":"The Appium Events API can be useful for getting a clear look at what's happening underneath the hood of your test, along the dimension of time. Using the Appium Event Parser, you can also generate a timeline to visualize events within Appium itself and within your own test script.","liveAt":"2019-11-06 10:00","canonicalRef":"https://www.headspin.io/blog/using-the-appium-events-api"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/94-using-the-appium-events-api","slug":"94-using-the-appium-events-api","path":"/editions/94-using-the-appium-events-api","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0093.md","num":93,"rawMd":"\n\nAppium supports automation of all kinds of Android apps---not just native apps. You can use Appium to [automate websites using the Chrome browser on Android](https://appiumpro.com/editions/4), and also to [automate webview-based portions of hybrid apps](https://appiumpro.com/editions/17). On iOS Appium has to implement a bunch of custom logic to attach to Safari or hybrid apps, but on Android, our life is made much easier by the existence of [Chromedriver](https://chromedriver.chromium.org/). Chromedriver is an implementation of the WebDriver protocol by Google, which can control Chrome browsers on both desktop and Android platforms.\n\n### How Chromedriver works\n\nWhen you request a context switch into a webview, or when you start an Appium session using the `browserName` capability set to `Chrome`, Appium starts up Chromedriver for you automatically as a subprocess. Any standard commands your test client sends to Appium while Chromedriver is active get passed directly onto Chromedriver at that point, so that Chromedriver is effectively the component which is driving the automation of the website within the browser or webview.\n\nAll of this works transparently to you as the Appium user because both Appium and Chromedriver expose the W3C WebDriver protocol! This fact even allows Appium to do clever things like decide *not* to pass certain commands to Chromedriver and instead execute its own implementation of them. This is why you can still run certain `mobile:` commands while in a webview or browser context, for example. From the Appium developers' perspective, Chromedriver is also nice because it makes our job a lot simpler; rather than figure out how to implement fundamental automation of Chrome ourselves, we can rely on the tool that Google has provided (and Google is the producer of Chrome, so they are in the best position to make the most robust and stable tool!).\n\nThere is one downside of relying on Chromedriver, however, which is that new versions of Chromedriver are not compatible with older versions of Chrome. Starting with version 73, Chromedriver updated its versioning scheme to match the version of Chrome. Nowadays, what you want to ensure is that if you have version `XX` of Chrome on your device, you are using version `XX` of Chromedriver with Appium. But Appium releases contain only *one* version of Chromedriver--typically the latest version of Chromedriver which is out in the wild at the time of an Appium release. (So Appium 1.15.1, for example, comes bundled with Chromedriver `76.0.3809.126`). This is because we don't want Appium users to have to download their own version of Chromedriver and manage it separately--that'd be too many moving parts!\n\nBut what if you've upgraded to Appium 1.15.1, but the version of Chrome on your device is only at 75? Well, then you'll abruptly encounter an error that looks like this:\n\n```txt\nAn unknown server-side error occurred while processing the command.\nOriginal error: unknown error: Chrome version must be >= 76\n```\n\nUh oh! How do you resolve this error? There are a few different strategies.\n\n### Strategy 1: Update Chrome\n\nThe first strategy is pretty obvious. Don't you want Chrome to be at the latest version, too? If so, just update Chrome (you can usually find x86-based APKs for emulators with a bit of judicious Googling), and you'll be good to go. But if you want to make sure you're at an older version of Chrome, or if you can't update, or are automating a webview targeting a different version of Chrome, then read on...\n\n### Strategy 2: Tell Appium to use a different Chrome\n\nAppium downloads and installs the version of Chrome which it was published with, if you don't tell it otherwise. But you can tell it otherwise! There is a command-line flag you can pass to the Appium install process that allows you to install any version of Chrome you like:\n\n```bash\nnpm install appium --chromedriver_version=\"75.0.3770.140\"\n```\n\nWhen you run this command, you'll see Appium downloading the version you specified instead of the default. Of course, this version must exist at the Chromedriver downloads site in the expected location!\n\n### Strategy 3: Use your own Chromedriver(s)\n\nIf you are responsible for automating different devices with different versions of Chrome, you might want the Appium server not to use just one version of Chromedriver, but to use a number of versions, based on the automation scenario. If you're in this situation, you can maintain your own local repository of Chromedriver executables, and pass the path to one of them in when you start the Appium server:\n\n```bash\nappium --chromedriver-executable /path/to/my/chromedriver\n```\n\n### Strategy 4: Let Appium download new Chromedrivers on the fly\n\nBy far the easiest of all the strategies is to just let Appium decide what version of Chromedriver works with your browser or webview, and download it when necessary for you. To make this work, you need to use a couple desired capabilities:\n\n* `chromedriverExecutableDir`: the path to a writable directory on the Appium server's host, where new Chromedriver binaries can be downloaded and executed from\n* `chromedriverChromeMappingFile`: the path to a JSON file on the Appium server's host, where a mapping will be stored of Chromedriver versions to Chrome support.\n\nNow, because Appium is downloading executables to the machine from the Internet, there's a potential security risk here (even though Appium does verify the Google-provided checksum of the downloaded files). What this means is to use this feature, you need to enable it as an \"insecure\" feature when you start the Appium server:\n\n```bash\nappium --allow-insecure chromedriver_autodownload\n```\n\n(If you have other insecure features to allow, like `execute_driver_script`, you can simply separate them with the `,` symbol).\n\nWhen you put all of this together, Appium will automatically download a Chromedriver to work with your webview or browser, and keep it in the directory specified, so that future requests for automating the same version of Chrome don't trigger a new download. So if you wind up in a case where you can't or don't want to upgrade Chrome, or are dealing with a potentially large number of devices with varying versions of Chrome on them, this strategy might be the key to some momentary sanity.\n","metadata":{"lang":"All Languages","platform":"Android","subPlatform":"All Devices","title":"Managing Chromedriver for Android Chrome and Webview Testing","shortDesc":"Appium uses Chromedriver to automate Android web and hybrid apps. Chromedriver is great, but it comes with its own wrinkles in terms of Chrome version support. In this edition, we take a look at four strategies for approaching these challenges.","liveAt":"2019-10-30 10:00","canonicalRef":"https://www.headspin.io/blog/managing-chromedriver-for-android-chrome-and-webview-testing"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/93-managing-chromedriver-for-android-chrome-and-webview-testing","slug":"93-managing-chromedriver-for-android-chrome-and-webview-testing","path":"/editions/93-managing-chromedriver-for-android-chrome-and-webview-testing","news":null,"tags":["All Languages","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0092.md","num":92,"rawMd":"\n\n

\n \"Birds\n

\n\nIt has been possible for some time to [use AI to find elements in Appium](https://appiumpro.com/editions/39), via the collaboration between [Test.ai](https://test.ai) and the Appium team. We work together to maintain the [test-ai-classifier](https://github.com/testdotai/appium-classifier-plugin) project, and in this edition of Appium Pro we'll take a look at an advanced element finding mode available in v3 of the plugin.\n\n### A tale of two modes\n\nThe default mode for the plugin is what I call \"element lookup\" mode. Let's say we're trying to find an icon that looks like a clock on the screen. The way this mode works is that Appium finds every leaf-node element in the view, generates an image of how it appears on screen, and then checks to see whether that image looks like a clock. If so, then the plugin will remember the element that was associated with that image, and return it as the result of the call to `findElement`.\n\nThis mode works great for finding many icons, and is pretty speedy on top of that. However, it can run into some big problems! Let's say we're working with the Files app on iOS:\n\n

\n \"The\n

\n\nNow let's say we want to find and tap on the \"Recents\" button at the bottom of the screen. There's an icon that looks like a clock, so we might be tempted to try and find the element using that label:\n\n```java\ndriver.findElement(MobileBy.custom(\"ai:clock\")).click();\n```\n\nThis might work depending on the confidence level set, but it will probably fail. Why would it fail? The issue is that, on iOS especially, the clock icon is not its own element! It's grouped together with the accompanying text into one element, like this:\n\n

\n \"The\n

\n\nWhen a screenshot is taken of this element, and it's fed into the classifier model, the arbitrary text that's included as part of the image will potentially confuse the model and lead to a bad result.\n\n### Object detection mode\n\nVersion 3.0 of the classifier plugin introduced a second element finding mode to deal with this kind of problem. Because there are now two modes, we need a way to tell the plugin which one we want to use. We do that with the `testaiFindMode` capability, which can take two values:\n\n* `element_lookup` (the default)\n* `object_detection`\n\nWhat is object detection, and how does it help? Object detection is a feature of many image-based machine learning models, which aims to take an input image and generate as its output a list of regions within that image that count as distinct objects. The image of birds with boxes around them up top is an example of what an object detection algorithm or machine learning model can achieve. Carving up the world into separate objects is something our human brains do quite naturally, but it's a fairly complex thing to figure out how to train a computer to do it!\n\nOnce objects have been detected in an image, they can optionally be classified. The object detection mode of the plugin does exactly this: it takes a screenshot of the *entire* screen, runs this *entire* screenshot through an object detection model, and then cuts up screen regions based on objects detected by the model. Those cut-up images (hopefully representing meaningful objects) are then sent through the same classifier as in element lookup mode, so we can see if any of them match the type of icon that we're looking for.\n\nThe main advantage of object detection mode is that the plugin will look at the screen the way a human would--if you ask it to find a clock, it won't matter if there's a unique 'clock' element all by itself somewhere on the screen or not. This is also the main disadvantage, because when using this mode, we are not finding actual elements. We're just finding locations on the screen where we believe an element to be, i.e., [image elements](https://appiumpro.com/editions/32). This is totally fine if we just want to tap something, but if we're trying to find an input box or other type of control to work with using something like `sendKeys`, we're out of luck.\n\n### Example\n\nBy way of example, I've written up a simple test case that attempts to tap on the \"Recents\" button in the Files app on iOS, using object detection. It's pretty slow (the object detection model takes about 30 seconds to run on my machine), so it's really only useful in situations where absolutely nothing else works at this point! I'm very grateful to Test.ai for contributing the object detection model which is downloaded in v3 of the plugin.\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.MobileElement;\nimport io.appium.java_client.Setting;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition092_AI_Object_Detection {\n\n private String BUNDLE_ID = \"com.apple.DocumentsApp\";\n private IOSDriver driver;\n private WebDriverWait wait;\n\n private By recents = MobileBy.custom(\"ai:clock\");\n private By browse = MobileBy.AccessibilityId(\"Browse\");\n private By noRecents = MobileBy.AccessibilityId(\"No Recents\");\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n caps.setCapability(\"platformVersion\", \"11.4\");\n caps.setCapability(\"deviceName\", \"iPhone 8\");\n caps.setCapability(\"bundleId\", BUNDLE_ID);\n\n HashMap customFindModules = new HashMap<>();\n customFindModules.put(\"ai\", \"test-ai-classifier\");\n\n caps.setCapability(\"customFindModules\", customFindModules);\n caps.setCapability(\"testaiFindMode\", \"object_detection\");\n caps.setCapability(\"testaiObjectDetectionThreshold\", \"0.9\");\n caps.setCapability(\"shouldUseCompactResponses\", false);\n\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testFindElementUsingAI() {\n driver.setSetting(Setting.CHECK_IMAGE_ELEMENT_STALENESS, false);\n\n // click the \"Browse\" button to navigate away from Recents\n wait.until(ExpectedConditions.presenceOfElementLocated(browse)).click();\n\n // now find and click the recents button by the fact that it looks like a clock\n driver.findElement(recents).click();\n\n // prove that the click was successful by locating the 'No Recents' text\n wait.until(ExpectedConditions.presenceOfElementLocated(noRecents));\n }\n}\n```\n\n(Of course, you can also check out the sample code [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition092_AI_Object_Detection.java))\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Using AI-based Object Detection For Finding Elements","shortDesc":"The AI-based element finding plugin for Appium now supports another strategy, namely finding all potentially matching icons on a screen, regardless of whether they map directly to elements.","liveAt":"2019-10-23 10:00","canonicalRef":"https://www.headspin.io/blog/using-ai-based-object-detection-for-finding-elements"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/92-using-ai-based-object-detection-for-finding-elements","slug":"92-using-ai-based-object-detection-for-finding-elements","path":"/editions/92-using-ai-based-object-detection-for-finding-elements","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0090.md","num":90,"rawMd":"\n\nIf you're automating an app which doesn't have standard UI elements that are easily findable using Appium's [standard locator strategies](https://appiumpro.com/editions/60) (for example, if you're automating a game), then you might have checked out the \"find by image\" feature that has proven useful in recent versions of Appium. (If you're unfamiliar with finding elements by image and want a thorough introduction, check out the Appium Pro editions on [finding elements by image, part 1](https://appiumpro.com/editions/32) and [part 2](https://appiumpro.com/editions/33)). In that series, I teased the intro with this image:\n\n

\n \"Angry\n

\n\nWe didn't actually automate Angry Birds back then, but we will today! To navigate through the first level of this classic game, we'll need to find and interact with 2 screen regions, and then assert our victory by matching a third region. Here are the images we will use for the match:\n\n* **checkmark.png** - this is a representation of the checkmark we'll need to tap to get from the mini tutorial to the first level, after the game has loaded for the first time.
 
\n* **red-bird-in-slingshot.png** - this is the red bird sitting in its slingshot, that we will need to construct an action around in order to shoot it at some (presumably deserving) pigs.
 
\n* **level-cleared-three-stars.png** - this is the image which is shown when we complete a level with flying colors, having destroyed all the pigs with some of our intrepid birds in reserve. We need to find this image to assert that our script has actually beaten the level.
 
\n\nNow, the way we send these images into Appium to look for matches is in Base64-encoded string format, so we need some helper methods to turn a local image path into a Base64-encoded string. I'm storing the images in a \"queryimages\" folder within the Appium Pro project, so my helper functions look like this:\n\n```java\nprivate File getImageFile(String imageName) {\n return new File(\"queryimages/\" + imageName + \".png\");\n}\n\nprivate String getReferenceImageB64(String imageName) throws IOException {\n Path refImgPath = getImageFile(imageName).toPath();\n return Base64.getEncoder().encodeToString(Files.readAllBytes(refImgPath));\n}\n```\n\nWith this in hand, I would normally be able to just find and tap on the first image (the checkmark) with just a few lines of code:\n\n```java\nString imageData = getReferenceImgB64(\"checkmark\");\ndriver.findElement(MobileBy.image(imageData)).click();\n```\n\nThis is great for the checkmark, but what about the red bird in the slingshot? We don't want to tap it; we want to tap, drag, and release it at just the right angle to destroy some stuff. To that end, I wrote another helper function entitled `shootBird` (the horrible things I do for you readers!), which takes the found bird element and constructs the appropriate low-level action using it:\n\n```java\nprivate void shootBird(AndroidDriver driver, WebElement birdEl, int xOffset, int yOffset) {\n Rectangle rect = birdEl.getRect();\n Point start = new Point(rect.x + rect.width / 2, rect.y + rect.height / 2);\n Point end = start.moveBy(xOffset, yOffset);\n Duration dragDuration = Duration.ofMillis(750);\n\n PointerInput finger = new PointerInput(Kind.TOUCH, \"finger\");\n Sequence shoot = new Sequence(finger, 0);\n shoot.addAction(finger.createPointerMove(Duration.ofMillis(0), Origin.viewport(), start.x, start.y));\n shoot.addAction(finger.createPointerDown(MouseButton.LEFT.asArg()));\n shoot.addAction(finger.createPointerMove(dragDuration, Origin.viewport(), end.x, end.y));\n shoot.addAction(finger.createPointerUp(MouseButton.LEFT.asArg()));\n driver.perform(Arrays.asList(shoot));\n}\n```\n\nThere's a lot of code there, but it's fundamentally pretty simple: we just find the mid-point of the bird element and construct what is essentially a drag-and-drop action using that mid-point.\n\nFinally, all we need to do to verify the victory is to check for the presence of our \"stars\" image:\n\n```java\nimageData = getReferenceImgB64(\"level-cleared-three-stars\");\ndriver.findElement(MobileBy.image(imageData));\n```\n\nIn a perfect world, just this code would be enough to reliably beat the first level of Angry Birds. If you run it as is, however, you might encounter some \"Element Not Found\" errors. This is often because the default image match threshold set by Appium may not be appropriate for the particular image you want to find.\n\nThe threshold is a number between 0 and 1 that tells Appium how strict you want it to be in finding a match. If you use a threshold of 1, you are telling Appium to find a perfect pixel-for-pixel match (which will almost never occur in a real-world situation). If you're using a threshold of 0, you are telling Appium that _any_ screen region is a good match---but that will likely result in false positive matches!\n\nThe question becomes, how do you know what threshold to use? As a general rule of thumb, I like a number in the neighborhood of `0.43` to `0.48`. Just kidding! Well, not really. I do find that range useful---but you should never just take my word for it. What we should do is determine the threshold experimentally, for our particular set of images. We can do this during test development by creating a wrapper function for `findElement`, that performs a binary search over the threshold space, to figure out the highest threshold we can expect to find the image at.\n\nIn a \"production\" environment, we could do a lot of things with this threshold data. We could store it in a database along with the image templates themselves, so that a helper function can automatically update the threshold when we try to find different images. We could use it as a \"baseline\", à la baseline images in visual testing. But for now, we're just going to print it out to the console so we as test developers can use this information to adjust our thresholds.\n\n(If you've never encountered or don't remember the concept of a [binary search](https://en.wikipedia.org/wiki/Binary_search_algorithm), now might be a good time to have a quick refresher). I call my threshold finder function `findImageWithOptimizationNotes`, and it looks like this:\n\n```java\nprivate WebElement findImageWithOptimizationNotes(String imageName) throws Exception {\n String imageData = getReferenceImageB64(imageName);\n WebElement el = null;\n double max = 1.0;\n double min = 0.0;\n double haltSearchSpread = 0.05;\n double check = 0;\n NotFoundException notFound = null;\n\n while (Math.abs(max - min) > haltSearchSpread) {\n check = (max + min) / 2;\n driver.setSetting(Setting.IMAGE_MATCH_THRESHOLD, check);\n try {\n el = driver.findElement(MobileBy.image(imageData));\n min = check;\n } catch (NotFoundException err) {\n max = check;\n notFound = err;\n }\n }\n\n if (el != null) {\n System.out.println(\"Image '\" + imageName + \"' was found at the highest threshold of: \" + check);\n return el;\n }\n\n System.out.println(\"Image '\" + imageName + \"' could not be found even at a threshold as low as: \" + check);\n throw notFound;\n}\n```\n\nBasically, we pass in the name of an image we want to find, and it uses a binary search to determine the highest threshold we can safely use to guarantee finding that image. (To keep the search from falling into [Zeno's paradox](https://en.wikipedia.org/wiki/Zeno%27s_paradoxes), we stop when the binary search space becomes smaller than `0.05` threshold units). Here's how this function can be used:\n\n```java\n@Test\npublic void testPigDestruction() throws Exception {\n Thread.sleep(12000);\n findImageWithOptimizationNotes(\"checkmark\").click();\n\n Thread.sleep(3000);\n WebElement birdEl = findImageWithOptimizationNotes(\"red-bird-in-slingshot\");\n shootBird(driver, birdEl, -280, 140);\n\n Thread.sleep(14000);\n findImageWithOptimizationNotes(\"level-cleared-three-stars\");\n}\n```\n\nYou can see that `findImageWithOptimizationNotes` essentially replaces `driver.findElement` in my test. You might be wondering, though, why the `Thread.sleep`s all over the place? Isn't that a bad practice? Yes, when we are running tests. However, we're not really running a test at this phase; we're using our test framework to help us _write_ tests. And in this context, we don't want to use explicit or implicit waits, otherwise our binary search will be forced to waste a lot of time, or we'll kick the search off before we know the element is actually visible. In this case, I've used my knowledge of how long it takes the game to transition between various screens in order to ensure that when the binary search starts, the image is displayed on screen. Needless to say, you'd want to rewrite this for an actual test that lives in CI.\n\nCurious what thresholds worked for me? Here's what I got when I ran the code on my Android emulator:\n\n```\nImage 'checkmark' was found at the highest threshold of: 0.46875\nImage 'red-bird-in-slingshot' was found at the highest threshold of: 0.59375\nImage 'level-cleared-three-stars' was found at the highest threshold of: 0.53125\n```\n\nWhat these results mean is that I could set the threshold once at around `0.45`, or I could reset it individually for the various elements according to their findability. Note that you might receive different values, based on differences in screen size or device. Working with image finding _across_ devices, DPIs, and screen sizes is a whole other topic which we'll certainly cover in a future edition of Appium Pro.\n\nFeel free to check out the [full code sample on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition090_Image_Element_Optimization.java), but keep in mind that you'll need to have the Angry Birds APK installed on your Android device (since I'm pretty sure it's not legal for me to distribute it). Have fun!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Optimizing Image Element Thresholds","shortDesc":"Finding elements via a template image is a powerful technique for automating apps with non-standard UI elements, as well as games. It can be tricky, though, to know how to appropriately use the threshold setting for finding image elements. In this edition we take a look at a technique for setting this threshold experimentally.","liveAt":"2019-10-09 10:00","canonicalRef":"https://www.headspin.io/blog/optimizing-image-element-thresholds"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/90-optimizing-image-element-thresholds","slug":"90-optimizing-image-element-thresholds","path":"/editions/90-optimizing-image-element-thresholds","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0089.md","num":89,"rawMd":"\n\nAppium is not just one \"thing\". It can automate multiple platforms, from iOS to Android and beyond. The way that Appium organizes itself around this multi-platform model is by means of various \"drivers\". This is more or less the same architecture as was first adopted by Selenium/WebDriver, which also utilizes a number of independent \"drivers\" in order to support the automation of multiple browsers.\n\nThere is one Appium driver per underlying automation technology. This almost means one Appium driver per platform (one for iOS, one for Android, etc...), but not quite. This is because some platforms (like Android) have multiple automation technologies which Appium targets to support automation of that platform. Android actually has 3 Appium drivers: one based on UiAutomator, one based on UiAutomator 2, and one based on Espresso.\n\n### How Drivers Work\n\nThe driver is arguably the most important concept in all of Appium. It's ultimately the driver's responsibility to turn the Appium API (known as the WebDriver Protocol) into automation for a particular platform. Drivers are essentially translators that turn Appium client commands into ... something else---whatever gets the job done on the platform!\n\nFor architectural simplicity among other reasons, each individual driver is itself a standalone WebDriver-compatible server (though it doesn't have all the options the main Appium server does).Within the driver, commands that are received are handled in idiosyncratic ways. They might be passed on to a separate process running as a Java program on an Android device, for example.\n\nDrivers themselves can have quite a complex internal architecture, sometimes relying on a whole stack of technologies. Here's a diagram showing the full stack of technologies involved in the XCUITest driver (the current iOS driver):\n\n

\n \"The\n

\n\nThere's quite a lot going on! The XCUITest driver is made available as part of Appium, and is brought to life whenever someone starts an iOS session. Internally, it spins up another bit of technology known as WebDriverAgent, which is responsible for turning WebDriver protocol commands into XCUITest library calls.\n\nMany drivers have an architecture like this, though each driver can set up its architecture however it likes, so long as, at the end of the day, it is published as an NPM package that exposes a class which extends Appium's `BaseDriver` class. This is what makes it possible for a driver to plug easily into Appium!\n\n### The Drivers\n\nWell, what drivers _are_ there? It's honestly a bit hard to say exactly, because there exist \"unofficial\" drivers in addition to the ones that ship with Appium. But if we take a look at the current list in the Appium codebase itself, we can see a fair few:\n\n```js\nconst AUTOMATION_NAMES = {\n APPIUM: 'Appium',\n UIAUTOMATOR2: 'UiAutomator2',\n UIAUTOMATOR1: 'UiAutomator1',\n XCUITEST: 'XCUITest',\n YOUIENGINE: 'YouiEngine',\n ESPRESSO: 'Espresso',\n TIZEN: 'Tizen',\n FAKE: 'Fake',\n INSTRUMENTS: 'Instruments',\n WINDOWS: 'Windows',\n MAC: 'Mac',\n};\n```\n\nThese \"automation names\" are the labels given to the various drivers which Appium knows about. This bit of code defines which strings are allowed to be used as values for the `automationName` capability. Of course, each driver typically only supports one platform. Here's a brief description of each of the drivers, by their `automationName`:\n\n* `Appium`: this automation name really means \"just give me the default driver for the platform I've chosen.\" It's not actually a separate driver on its own.\n* `UiAutomator2` ([repo](https://github.com/appium/appium-uiautomator2-driver)): this is the current default Android driver, based on Google's UiAutomator technology.\n* `UiAutomator1` ([repo](https://github.com/appium/appium-android-driver)): this is the older Android driver, based on an older version of UiAutomator.\n* `XCUITest` ([repo](https://github.com/appium/appium-xcuitest-driver)): this is the current iOS driver, based on Apple's XCUITest technology.\n* `YouiEngine` ([repo](https://github.com/YOU-i-Labs/appium-youiengine-driver)): this is a driver produced by You.i Labs, to support automation of apps on many different platforms built using their SDK.\n* `Espresso` ([repo](https://github.com/appium/appium-espresso-driver)): this is the newest Android driver, based on Google's Espresso technology.\n* `Tizen` ([repo](https://github.com/Samsung/appium-tizen-driver)): this is a driver produced by Samsung to assist in automation of Xamarin apps built for the Tizen OS.\n* `Fake`: the \"fake\" driver is used internally by Appium for the purpose of testing, and you shouldn't need to ever use it!\n* `Instruments` ([repo](https://github.com/appium/appium-ios-driver)): this is an older iOS driver based on an Apple technology which was removed after iOS 9. Basically, don't use this!\n* `Windows` ([repo](https://github.com/appium/appium-windows-driver)): Microsoft put together an Appium-compatible server called [WinAppDriver](https://github.com/microsoft/WinAppDriver), and this is the driver that connects it up with the main Appium server. You can use this driver to automate Windows apps!\n* `Mac` ([repo](https://github.com/appium/appium-mac-driver)): this is a driver which enables automation of Mac desktop apps.\n\nAs mentioned above, each of these drivers has its own internal architecture, as you can see in this detailed diagram:\n\n

\n \"All\n

\n\n### Drivers FAQ\n\n*How do I know which driver to use?* Well, if you want to automate iOS, Windows, Mac, or Tizen, your choice is simple: use the only driver which currently enables automation of that platform! If you want to use Android, you have the choice of the UiAutomator2 driver or the Espresso driver. It's worth learning a bit about each of these technologies to see which one might better support your use case. The feature set for these drivers is similar but not identical.\n\n*Do all the drivers support the same commands in the same way?* Yes and no. At a certain fundamental level, we are limited by the automation capabilities provided by the platform vendors. A \"tap\" on an Android device is the same as the \"tap\" on an iOS device. But other commands might not work in exactly the same way. As far as possible the Appium team tries to ensure parity of behavior across platforms and drivers.\n\n*Can I switch from one driver to another and expect my tests to pass?* Yes and no. It all depends on which drivers we're talking about. Part of the benefit of using Appium is that you _can_ change from one automation technology (like UiAutomator2) to another (like Espresso) without throwing away your entire test suite. But you should perform the migration slowly and methodically, making sure that everything is happening as you expect. The Appium team sometimes publishes migration guides for moving from one driver to another; check those out if possible!\n\n*Can I make my own driver?* Yes! Lots of people have done this, most recently both [Jason Huggins](https://github.com/hugs/appium-arduino-driver) and [myself](https://github.com/jlipps/appium-raspi-driver) (at AppiumConf 2019). But there are others too, like Christian Bromann's [hbbtv-driver](https://github.com/christian-bromann/appium-hbbtv-driver).\n\n*Will anything change with drivers in Appium 2.0?* I'm so glad you asked! One unwieldy bit of Appium's driver system is that we have to include drivers as strict dependencies of the Appium server. But we want the drivers to exist in more of a loosely-related ecosystem, where you can pick and choose which drivers you want to use with Appium. This means that you won't need to install the old UiAutomator2 driver and its dependencies if all you are using Appium for is running iOS tests! (Did you know there is a [proposal for Appium 2.0's design](https://gist.github.com/jlipps/651b62316603400cabc95ff0f9faf70f) out there on the Internet?)\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Understanding Appium Drivers (And How To Choose Which One To Use)","shortDesc":"Appium supports a whole host of platforms by means of a large array of 'drivers', each of which is responsible for translating Appium's own protocol to automation behaviors on a particular platform, using a particular automation technology. Learn all about this system, and how to choose the right driver for your testing.","liveAt":"2019-10-02 10:00","canonicalRef":"https://www.headspin.io/blog/understanding-appium-drivers-and-how-to-choose-which-one-to-use"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/89-understanding-appium-drivers-and-how-to-choose-which-one-to-use","slug":"89-understanding-appium-drivers-and-how-to-choose-which-one-to-use","path":"/editions/89-understanding-appium-drivers-and-how-to-choose-which-one-to-use","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0088.md","num":88,"rawMd":"\n\nWhen we're writing tests on our local machines, we're usually running just one Appium test at a time, with the server log window to one side to help us in case something goes wrong. We're also usually watching the test execute on the simulator or device (unless we've decided to take a coffee break at that moment). In this type of scenario, it's often pretty easy to diagnose a test failure, because we can see immediately that the test didn't proceed to the correct view, find the correct element, or whatever the issue might be.\n\n### Debugging in CI\n\nThis happy little scenario, however, is not typically the one we find ourselves when we actually _need_ to debug something. We never commit failing tests to our build (right? right?), so when there's a build failure because of one of our tests, it's because something went wrong that we didn't expect. In this case, it's usually not just one test that ran, but many, and by the time we know of the failure, the test itself is long finished. At this point, we can't watch the test run, and unless we did something special to save it, we no longer have access to the Appium logs or any record of the app state at the time of failure.\n\nSo let's do that special something to make sure that we don't wind up in a horrible debugging situation with nothing to go on!\n\n### Capturing Debug Helpers\n\nThe basic idea here is that we want to inject some code into the test lifecycle, specifically in between test failure and cleaning up to move on to the next test. At this point in the test lifecycle, we know we've got a failed test, but we haven't yet quit our Appium session, so we still have access to a `driver` object. This is great, because (assuming the Appium session itself is still alive and working correctly), we can use it to retrieve all kinds of helpful diagnostic information about the session, which we can then save on disk for for later review should the need arise.\n\nWhat do we want to save? That is of course up to us, but I'm proposing a basic set of 3 useful items:\n\n1. The Appium server logs --- these contain the log of everything Appium thinks it's doing with your session, so it's a great place to debug issues that might have something to do with Appium itself, or to go back and make sure everything happened as expected in the earlier phases of the session. If you're not sure what to do with the Appium server logs, or how to read them, check out [Isaac Murchie's great AppiumConf 2018 talk](https://www.youtube.com/watch?v=mOkgAxIWdGg) on the subject.\n2. The app source --- especially if your error is of the \"element not found\" category, having the app/page source handy while investigating a test failure can be a great sanity check.\n3. A screenshot --- nothing beats seeing exactly what was on the device screen when your test failed. Was it an errant pop-up? Had the app crashed and gone away all together? A visual inspection can sometimes reveal the true cause of a test failure much more quickly than a scan of logs.\n\nThese are all in addition, of course, to the client/test log itself, which is output within the CI system and therefore easy to save one way or another.\n\nTo actually retrieve all this data, we simply need to use the appropriate Appium commands. Then it's up to us what to do with them. In this example, I'll just write them to stdout, assuming that all logging from the test output is being saved somewhere reasonable:\n\n```java\n// print appium logs\nLogEntries entries = driver.manage().logs().get(\"server\");\nSystem.out.println(\"======== APPIUM SERVER LOGS ========\");\nfor (LogEntry entry : entries) {\n System.out.println(new Date(entry.getTimestamp()) + \" \" + entry.getMessage());\n}\nSystem.out.println(\"================\");\n\n// print source\nSystem.out.println(\"======== APP SOURCE ========\");\nSystem.out.println(driver.getPageSource());\nSystem.out.println(\"================\");\n\n// save screenshot\nString testName = desc.getMethodName().replaceAll(\"[^a-zA-Z0-9-_\\\\.]\", \"_\");\nFile screenData = driver.getScreenshotAs(OutputType.FILE);\ntry {\n File screenFile = new File(SCREEN_DIR + \"/\" + testName + \".png\");\n FileUtils.copyFile(screenData, screenFile);\n System.out.println(\"======== SCREENSHOT ========\");\n System.out.println(screenFile.getAbsolutePath());\n System.out.println(\"================\");\n} catch (IOException ign) {}\n```\n\nOnce you strip away the gratuitous log printing, you'll see that the code above is pretty straightforward. The only complexity comes in retrieving the Appium server logs. On the client side, we use the log manager to retrieve logs of type `server`. Because Appium servers might plausibly handle sessions for multiple clients at a time, however, server log retrieval is not enabled by default as a potential information security risk.\n\nIf you run your own instance of Appium, you can turn on server log retrieval using the `--allow-insecure` flag:\n\n```\nappium --allow-insecure=get_system_logs\n```\n\nThis will direct Appium not to complain if you attempt to get the system logs!\n\n### JUnit's TestWatcher\n\nThe question remains, though: where to put the code above? Each test framework has a different way of figuring out whether a test has failed and performing some actions before moving on to the next test. In JUnit, this is via the `TestWatcher` class. The way we use it is to create our own extension of this class, and override particular methods of `TestWatcher` pertaining to the aspects of the lifecycle where we want to inject code. Then we assign our new watcher class as a `@Rule` in our test class itself. From a skeletal perspective, it would look something like the following:\n\n```java\npublic class Edition088_Debugging_Aids {\n private IOSDriver driver;\n\n @Rule\n public DebugWatcher debugWatcher = new DebugWatcher();\n\n @Before\n public void setUp() throws MalformedURLException { /*...*/ }\n\n @Test\n public void testSomething() { /*...*/ }\n\n public class DebugWatcher extends TestWatcher {\n\n @Override\n protected void failed(Throwable e, Description desc) {\n // ---> this is where our debug print lines go! <---\n }\n\n @Override\n protected void finished(Description desc) {\n if (driver != null) {\n driver.quit();\n }\n }\n }\n}\n```\n\nWe can, as in the example above, define our watcher class right in our test class itself, if we want (but we don't have to). The important things are to (a) override the `failed` method and put all our debug handler code in it, and (b) override the `finished` method to act as our tear-down function, wherein we clean up the driver and anything else we need.\n\nThat's basically it! It's up to you to get creative with where you put the debug helper info retrieved from Appium--send it to an asset storage system? Build your own debug UI on top of it? Whatever you want.\n\nMake sure to take a look at the [full example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition088_Debugging_Aids.java) on GitHub, with actual test logic filled out, so you can see how everything works when it's fully put together. Happy debugging (or at least, happi_er_ debugging)!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Saving Test Data To Make Debugging Easier","shortDesc":"It's all too common to end up in a position where you're trying to debug a test that was run remotely, without access to any of the data that would make debugging possible. Learn how not to end up in this position by setting your test cases to automatically retrieve and save useful debugging information on test failure.","liveAt":"2019-09-25 10:00","canonicalRef":"https://www.headspin.io/blog/saving-test-data-to-make-debugging-easier"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/88-saving-test-data-to-make-debugging-easier","slug":"88-saving-test-data-to-make-debugging-easier","path":"/editions/88-saving-test-data-to-make-debugging-easier","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0087.md","num":87,"rawMd":"\n\nAndroid 10, by and large, works the same way with Appium as any previous version. There is one important change to be aware of from an automation perspective, however. If the app you're testing has a target SDK version of less than 23 (Android 6.0), then when you launch it on Android 10, you might see a fancy new permissions interstitial before your app launches. Something like this:\n\n

\n \"Permissions\n

\n\nPresumably this is because of Android 10's new, more fine-grained app permissions model. Because these older apps haven't been coded with an eye to the new permissions model, Android needs to pop up this annoying interstitial in order to make sure everything's cool, permissions-wise. This is a slight annoyance for users, with a great gain in terms of privacy. However, it can be devastating for automation if your code isn't expecting it. You have three options for what to do about this.\n\n### Option 1: Automate the Permissions View\n\nThe first option is the big hammer: you could just use Appium to automate this dialog by tapping the \"Continue\" button. But this means you have to throw a conditional in your test code for Android 10, and potentially waste a bunch of timing waiting for a \"Continue\" button that doesn't exist on versions of Android less than 10. So I don't recommend it.\n\n### Option 2: Update your app's targetSdkVersion\n\nEvery Android app can specify 3 different Android SDK versions. Somewhat confusingly, they are:\n\n* `minSdkVersion`: This is the minimum version of Android your app will launch on.\n* `compileSdkVersion`: This is the version of Android that your app will be built with, which determines which APIs your code has access to as you are compiling it. It also determines which compilation errors and warnings will be logged.\n* `targetSdkVersion`: This is kind of a weird one. By targeting a certain SDK, you are telling Android which SDK you have tested, so Android is free to apply system-wide behavior changes to your app. For example, imagine that your app has a `targetSdkVersion` of 21---in that case, the Android OS might decide that your app doesn't support Dark Mode, even though Dark Mode is available more generally on the device which is running your app. Essentially, it clues Android in to which system behaviors your app is designed to support.\n\nIf you have access to your app's source code, or can convince your developers to do this, you could always update your app's `targetSdkVersion` to 29 (the SDK version of Android 10), and this will do away with the interstitial. Though of course you'll need to participate in the new permissions system, and that might require some code updates throughout the app.\n\n### Option 3: Use `autoGrantPermissions`\n\nThe final, and in my opinion most straightforward, option is to use the `autoGrantPermissions` capability. If you include this capability and set it to `true`, Appium will attempt to automatically grant your app all permissions from the system perspective, so that when it comes time to launch your app, Android already thinks that all permissions have been requested and accepted.\n\nUsing this capability is nice because you might need it already (as a way to do away with dialogs popping up throughout the execution of your test), but also because you don't need to update the app in order to use it. Of course, you should still bug your developers to update the target SDK version for your app so that it stays current!\n\nAppium 1.15 is the most current release that supports Android 10, so make sure you download it whenever you need to automate apps on this Android platform. As of the time of writing, Appium 1.15 is currently in the Release Candidate stage, and should be generally available soon!\n","metadata":{"lang":"All Languages","platform":"Android","subPlatform":"All Devices","title":"Working With Android 10","shortDesc":"Android 10 comes with a few changes to the permissions model that might cause issues for app automation, especially of older apps. In this article we take a look at how to work around these changes and keep the automation rolling.","liveAt":"2019-09-18 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-android-10"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/87-working-with-android-10","slug":"87-working-with-android-10","path":"/editions/87-working-with-android-10","news":null,"tags":["All Languages","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0086.md","num":86,"rawMd":"\n\nThis edition of Appium Pro is in many ways the sequel to the earlier article on [how to batch Appium commands together using Execute Driver Script](https://appiumpro.com/editions/85). In that article, we saw one way of getting around network latency, by combining many Appium commands into one network request to the Appium server.\n\nWhen using a cloud service, however, there might be other network-related issues to worry about. Many cloud services adopt the standard Webdriver/Appium client/server model for running Appium tests. But because they host hundreds or thousands of devices, they'll be running a very high number of Appium servers. To reduce complexity for their users, they often provide a single entry point for starting sessions. The users' requests all come to this single entry point, and they are proxied on to the appropriate Appium server based on the user's authentication details and the session ID. In these scenarios, the single entry point acts as a kind of Appium load balancer, as in the diagram below:\n\n

\n \"Cloud\n

\n\nThis model is great for making it easy for users to connect to the service. But it's not necessarily so great from a test performance perspective, because it puts an additional HTTP request/response in between your test client and the Appium server which is ultimately handling your client's commands. How big of a deal this is depends on the physical arrangement of the cloud service. Some clouds keep their load balancers and devices all together within one physical datacenter. In that case, the extra HTTP call is not expensive, because it's local to a datacenter. Other cloud providers emphasize geographical and network distribution, with real devices on real networks scattered all over the world. That latter scenario implies Appium servers _also_ scattered around the world (since Appium servers must be running on hosts physically connected to devices). So, if you want both the convenience of a single Appium endpoint for your test script plus the benefit of a highly distributed device cloud, you'll be paying for it with a bunch of extra latency.\n\nWell, the Appium team really doesn't like unnecessary latency, so we thought of a way to fix this little problem, in the form of what we call _direct connect_ capabilities. Whenever an Appium server finishes starting up a session, it sends a response back to your Appium client, with a JSON object containing the capabilities the server provides (usually it's just a copy of whatever capabilities you sent in with your session request). If a cloud service implements direct connect, it will add four new capabilities to that list:\n\n* `directConnectProtocol`\n* `directConnectHost`\n* `directConnectPort`\n* `directConnectPath`\n\nThese capabilities will encode the location and access information for a _non-intermediary_ Appium server--the one actually handling your test. Now, your client had connected to the Appium load balancer, so it doesn't know anything about the host and port of the non-intermediary Appium server. But these capabilities give your client that information, and if your client also supports direct connect, it will parse these capabilities automatically, and ensure that each subsequent command gets sent not to the load balancer but directly to the Appium server which is handling your session. At this point in time, the official Appium Ruby and Python libraries support direct connect, as well as WebdriverIO--support for other clients coming soon.\n\nIt's essentially what's depicted in the diagram below, where for every command after the session initialization, HTTP requests are made directly to the final Appium server, not to the load balancer:\n\n

\n \"Cloud\n

\n\nThe most beautiful thing about this whole feature is that you don't even need to know about direct connect for it to work! It's a passive client feature that will work as long as the Appium cloud service you use has implemented it on their end as well. And, because it's a new feature all around, you _may_ have to turn on a flag in your client to signal that you want to use this feature if available. (For example, in WebdriverIO, you'll need to add the `enableDirectConnect` option to your WebdriverIO config file or object.) But beyond this, it's all automatic!\n\nThe only other thing you might need to worry about is your corporate firewall--if your security team has allowed connections explicitly to the load balancer through the firewall, but not to other hosts, then you may run into issues with commands being blocked by your firewall. In that case, either have your security team update the firewall rules, or turn off direct connect so your commands don't fail.\n\n## Direct Connect In Action\n\nTo figure out the actual, practical benefit of direct connect, I again engaged in some experimentation using [HeadSpin](https://ui.headspin.io/register?referral=start-testing-appiumpro)'s device cloud (HeadSpin helped with implementing direct connect, and their cloud currently supports it).\n\nHere's what I found when, from my office in Vancouver, I ran a bunch of tests, with a bunch of commands, with and without direct connect, on devices sitting in California and Japan (in all cases, the load balancer was also located in California):\n\n|Devices|Using Direct Connect?|Avg Test Time|Avg Command Time|Avg Speedup|\n|------|---------------------|-------------|----------------|-----------|\n|Cali|No|72.53s|0.81s||\n|Cali|Yes|71.62|0.80s|1.2%|\n|Japan|No|102.03s|1.13s||\n|Japan|Yes|70.83s|0.79s|30.6%|\n\n### Analysis\n\nWhat we see here is that, for tests I ran on devices in California, direct connect added only marginal benefit. It _did_ add a marginal benefit with no downside, so it's still a nice little bump, but because Vancouver and California are pretty close, and because the load balancer was geographically quite close to the remote devices, we're not gaining very much.\n\nLooking at the effects when the devices (and therefore Appium server) are located much further away, we see that direct connect provides a very significant speedup of about 30%. This is because, without direct connect, each command must travel from Vancouver to California and then on to Japan. With direct connect, we not only cut out the middleman in California, but we also avoid constructing another whole HTTP request along the way.\n\n### Test Methodology\n\n(The way I ran these tests was essentially the same as the way I ran tests for [the article on Execute Driver Script](https://appiumpro.com/editions/85))\n\n* These tests were run on real Android devices hosted by HeadSpin around the world on real networks, in Mountain View, CA and Tokyo, Japan.\n* For each test condition (location and use of direct connect), 15 distinct tests were run.\n* Each test consisted of a login and logout flow repeated 5 times.\n* The total number of Appium commands, not counting session start and quit, was 90 per test, meaning 1,350 overall for each test condition.\n* The numbers in the table discard session start and quit time, counting only in-session test time (this means of course that if your tests consist primarily of session start time and contain very few commands, then you will get a proportionally small benefit from optimizing using this new feature).\n\n## Conclusion\n\nYou may not find yourself in a position where you need to use direct connect, but if you're a regular user of an Appium cloud provider, make sure to check in with them to ask whether they support the feature and whether your test situation might benefit from the use of it. Because the feature needs to be implemented in the load balancer itself, it's not something that you can take advantage of by using open source Appium directly (although, it would be great if someone built support for direct connect as a Selenium Grid plugin!) Still, as use of devices located around the world becomes more common, I'm happy that we have at least a partial solution for eliminating any unnecessary latency.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Connecting Directly to Appium Hosts in Distributed Environments","shortDesc":"When using Appium with cloud services, we can sometimes be victims of too many network hops and proxies. However, if the cloud service supports the `directConnect` response capabilities, it can make our lives a lot easier.","liveAt":"2019-09-11 10:00","canonicalRef":"https://www.headspin.io/blog/connecting-directly-to-appium-hosts-in-distributed-environments"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/86-connecting-directly-to-appium-hosts-in-distributed-environments","slug":"86-connecting-directly-to-appium-hosts-in-distributed-environments","path":"/editions/86-connecting-directly-to-appium-hosts-in-distributed-environments","news":{"rawMd":"I'll actually be speaking about the topic of this newsletter as well as a few other topics TODAY on this [webinar](https://info.headspin.io/webinar/speeding-up-your-remote-appium-sessions). It's happening in just a few hours so please join me by [signing up here](https://info.headspin.io/webinar/speeding-up-your-remote-appium-sessions)!\n\nAs mentioned last week, Cloud Grey is exploring taking its world-class Appium training on the road to a city near you! Are you interested in attending one of our excellent 2-day workshops? Help us figure out where to start by [filling out this survey](https://cloudgrey.typeform.com/to/sItabH), which will close soon. Thanks in advance!\n","num":86,"mdPath":"/vercel/path0/content/news/0086.md"},"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0085.md","num":85,"rawMd":"\n\nBoth Appium and Selenium are based on a client/server architecture, where the test commands are triggered by a client that could be very far away from the server which actually performs the commands. The advantages of this client/server architecture are very important, allowing Appium tests to be written in any language, and making it possible to build large private or public Appium device clouds.\n\nThere is one significant downside to this architecture, however, that can dramatically decrease test performance. Because every test command has to travel over the network between the client and server, every command is subject to the latency of that network, as well as any other \"atmospheric\" conditions. Networks are not perfectly reliable in general, and unanticipated slowness or request failure can occur at any point. This means that Appium scripts developed locally might behave significantly differently when run on an Appium cloud somewhere across the world, leading to heightened flakiness or test failures.\n\nOn top of this, tests are not often written by directly implementing Appium commands, but more often by using framework-level functionality, which might encapsulate the use (and overuse) of many Appium commands, all of which add to test execution time. In some cases, I have seen test frameworks that make 5-10 requests to the server for every found element, in order to retrieve element metadata just in case it is useful later on. Apart from being a bad idea in general, this kind of approach can lead to dramatic differences of execution time when run in cloud environments.\n\nMore than general slowness, latency is also a killer for real-time automation. If you need to be certain that command B happens a very short time after command A, then sending command B across the global Internet is not going to deliver that guarantee. This is one reason the W3C WebDriver spec team decided to build the new Actions API in a form where the entire action _chain_ is encoded as a single API call, even though the chain might take seconds or minutes to actually execute once the action begins.\n\n## Execute Driver Script\n\nThe Appium team has now done the same thing, not just for actions, but for _any Appium commands at all_. Essentially, we have created a single Appium command that allows you to pack as many other Appium commands inside it as you want. All these commands will be executed _on the Appium server itself_, so will not be subject to network latency. How does this work? It's magic, obviously! Imagine we have this test script written in the Java client:\n\n```java\n@Test\npublic void testLoginNormally() {\n driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);\n driver.findElement(MobileBy.AccessibilityId(\"Login Screen\")).click();\n driver.findElement(MobileBy.AccessibilityId(\"username\")).sendKeys(\"alice\");\n driver.findElement(MobileBy.AccessibilityId(\"password\")).sendKeys(\"mypassword\");\n driver.findElement(MobileBy.AccessibilityId(\"loginBtn\")).click();\n driver.findElement(By.xpath(\"//*[@text='Logout']\")).click();\n}\n```\n\nAs you can probably tell, it's a very straightforward login/logout set of commands. If necessary, we could run all these commands in one go, as a batch, using the new `executeDriverScript` command:\n\n```java\n@Test\npublic void testLoginWithExecute() {\n driver.executeDriverScript(\n \"await driver.setImplicitTimeout(10000);\\n\" +\n \"await (await driver.$('~Login Screen')).click();\\n\" +\n \"await (await driver.$('~username')).setValue('alice');\\n\" +\n \"await (await driver.$('~password')).setValue('mypassword');\\n\" +\n \"await (await driver.$('~loginBtn')).click();\\n\" +\n \"await (await driver.$('//*[@text=\\\"Logout\\\"]')).click();\\n\"\n );\n}\n```\n\nWhat on earth is going on here? It looks like we've got some kind of Appium client code wrapped up in a string, somehow? That's right! The Appium team debated many ways of implementing this \"batch command\" feature, but at the end of the day decided that giving users complete flexibility in terms of what to run within the batch was of utmost importance. So we implemented this [Execute Driver Script](https://github.com/appium/appium/blob/master/commands-yml/commands/session/execute-driver.yml) command, where the command argument is a string representing JavaScript code to be executed in the context of the currently-running Appium session. Whatever you put in that string will be attempted to be executed by the Appium server.\n\nUh oh! Isn't that the definition of a remote code execution vulnerability? Yes! So we need to say a couple words about security. First, because there is no way to know what kind of junk a user might send in with this command, the server must be started in a special mode that allows this feature explicitly:\n\n```\nappium --allow-insecure=execute_driver_script\n```\n\nSecondly, all code is run within a NodeJS [VM](https://nodejs.org/api/vm.html), which means it does not share an execution context with the main Appium process. In fact, we can tightly control what methods the executing code has access to, and we give access to basically nothing except a `driver` object. What is this `driver` object? It's an instance of a [WebdriverIO](https://webdriver.io) session object. So you can use the entire WebdriverIO API, and all the JavaScript syntax your heart desires! This explains the interesting bits of the code above, like the `driver.$` method (which is WebdriverIO's equivalent of `findElement`), or the fact that accessibility ID locators are defined by putting `~` at the front. You can also _return_ text, data, or even elements inside your code string, and the result will be fully usable from within the parent script.\n\n## Execute Driver Script In Action\n\nI wanted to get a good idea of the impact of the Execute Driver Script on test execution times, so I ran a bunch of experiments on the only Appium cloud provider which currently supports this feature: [HeadSpin](https://ui.headspin.io/register?referral=start-testing-appiumpro). My test methodology is detailed below, but here are the results (in all cases, the client location is Vancouver, Canada):\n\n|Server|Using Execute Driver?|Avg Test Time|Avg Command Time|Avg Speedup|\n|------|---------------------|-------------|----------------|-----------|\n|Localhost|No|49.12s|0.55s||\n|Localhost|Yes|48.71s|0.54s|0.8%|\n|Mountain View, CA|No|72.53s|0.81s||\n|Mountain View, CA|Yes|43.15s|0.48s|40.5%|\n|Tokyo, Japan|No|102.03s|1.13s||\n|Tokyo, Japan|Yes|42.10s|0.47s|58.74%|\n\n### Analysis\n\nIn the case of local execution, use of Execute Driver Script does not deliver much of an improvement, and this is expected. When client and server are already located on the same network interface, there is basically no time lost to latency. What we see in the examples where the Appium server is located somewhere else in the world is much more drastic. Mountain View, CA is much closer to my office in Vancouver than Tokyo is, and that is reflected in the ~30% difference in the control case for each location. This difference is basically entirely due to latency, and highlights exactly the problem with the client/server model when deployed in this case--about 30 seconds per test, when the command count is high (in this case, 90 commands per test).\n\nWhen I adjust my script to use Execute Driver Script entirely, so that all 90 commands are contained within one batch, what we see is that test time is basically a low constant number across all environments. Since I'm just making one network call, latency due to geographic distribution becomes a negligible factor, reducing test behavior time by a factor of 40-60%! Of course, your results with this feature will vary greatly due to any number of factors, including the number of commands you put into the batch call, etc... I am also not recommending that every command be stuffed into one of these Execute Driver Script calls, merely demonstrating the performance improvements which might be relevant for a use case you encounter.\n\n### Test Methodology\n\n* These tests were run on real Android devices hosted by HeadSpin around the world on real networks, in Mountain View, CA and Tokyo, Japan. (Locally, the tests were run on an emulator and an Appium server running on my busy laptop, and thus should not be compared in absolute terms to the real devices.)\n* For each test condition (location and use of Execute Driver Script), 15 distinct tests were run.\n* Each test consisted of a login and logout flow repeated 5 times.\n* The total number of Appium commands, not counting session start and quit, was 90 per test, meaning 1,350 overall for each test condition.\n* The numbers in the table discard session start and quit time, counting only in-session test time (this means of course that if your tests consist primarily of session start time and contain very few commands, then you will get a proportionally small benefit from optimizing using this new feature).\n\n## Conclusion\n\nExecute Driver Script is a new Appium feature that is especially useful when running your tests in a distributed context. If a cloud server or device is located across the world from you, each command will take longer than it would if the server were close. The farther away the device, the longer your command will take. The administrator of such a distributed cloud can opt to turn on the Execute Driver Script feature in Appium, to allow their users to batch commands as a way of avoiding tons of unnecessary, latency-filled back-and-forth with the server. This gives users the advantage of a geographically distributed cloud (whether the user wants geographic distribution for its own sake or because that is simply where the devices and servers happen to be located), without the typical latency cost associated with it. Of course, this is an advanced feature that you should only use to solve specific problems!\n\nIf you want to see that Java code in the context of the full project, you can [check it out on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition085_Execute_Driver_Script.java). Other Appium clients also support this new command, including WebdriverIO (so you can have WebdriverIO-ception!)\n\nMany thanks to [HeadSpin](https://ui.headspin.io/register?referral=start-testing-appiumpro) for enabling the feature and giving me devices to play with so I could collect some hard data for this article.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Batching Appium Commands Using Execute Driver Script to Speed Up Tests","shortDesc":"Appium's client/server model has some disadvantages when it comes to encountering network legacy for each command. Appium has a way around this challenge, by allowing you to batch up commands into a group to be executed server-side.","liveAt":"2019-09-04 10:00","canonicalRef":"https://www.headspin.io/blog/batching-appium-commands-using-execute-driver-script-to-speed-up-tests"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/85-batching-appium-commands-using-execute-driver-script-to-speed-up-tests","slug":"85-batching-appium-commands-using-execute-driver-script-to-speed-up-tests","path":"/editions/85-batching-appium-commands-using-execute-driver-script-to-speed-up-tests","news":{"rawMd":"Did you like this deep dive into Appium command performance? I'll be talking more about Execute Driver Script as well as some other exciting updates in Appium test performance at a [webinar](https://info.headspin.io/webinar/speeding-up-your-remote-appium-sessions) I'm giving next week (Sep 11) with HeadSpin. Please join me by [signing up here](https://info.headspin.io/webinar/speeding-up-your-remote-appium-sessions)!\n\nIn other news, Cloud Grey is exploring taking its world-class Appium training on the road to a city near you! Are you interested in attending one of our excellent 2-day workshops? Help us figure out where to start by [filling out this survey](https://cloudgrey.typeform.com/to/sItabH).\n","num":85,"mdPath":"/vercel/path0/content/news/0085.md"},"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0084.md","num":84,"rawMd":"\n\n> This is a guest post written by longtime Appium Pro reader [Wim Selles](https://twitter.com/wswebcreation). Thanks for contributing back to the Appium community, Wim! Be sure to check out both of Wim's AppiumConf talks.\n\nRecently I wanted to explore the possibilities of using deep links with Appium on real devices and I stumbled upon [Edition 7](https://appiumpro.com/editions/7) of Appium Pro, called Speeding Up Your Tests With Deep Links. In that edition we learned how to use shortcuts to make execution speed less important by using deep links.\n\nIn order to demonstrate and use deep links together with Appium the following command was used:\n\n```\ndriver.get('theapp://login//')\n```\n\nDuring my research I found out that this works perfectly for Android emulators and iOS simulators, but it didn't work on real devices. On iOS, Siri was opened, which is the native behaviour for iOS (see [this issue](https://github.com/appium/appium/issues/10947) for more details).\nIn this edition, I want to show you a different method of triggering deep links that will work for emulators, simulators and real devices. Let's start with the easy part, which will be Android.\n\n> Note:\n> The code examples are in JavaScript and can be used with WebdriverIO V5. I think the steps and code are self-explanatory, which makes it easy to translate it to your favourite language / framework.\n\n## Android and deep linking\n\nAndroid has a specific mobile command to use deep linking and can be found [here](http://appium.io/docs/en/commands/mobile-command/#android-uiautomator2-only). We can use the deep link command in the following way:\n\n```js\ndriver.execute(\n 'mobile:deepLink',\n {\n url: \"\",\n package: \"\"\n }\n);\n```\n\nFor _The App_ we need the following deep link format:\n\n```\ntheapp://login//\n```\n\nBased on this format it will wake up the app, perform authentication of the requested username/password combination behind the scenes and jump the user to the logged-in area instantly.\nWhen the url, the Android package name of _The App_, and the command are combined, we get this for Android:\n\n```js\ndriver.execute(\n 'mobile:deepLink',\n {\n url: \"theapp://login/darlene/testing123\",\n package: \"io.cloudgrey.the_app\"\n }\n);\n```\n\nThis command will work for Android emulators and real devices, even when the app is already opened. Pretty easy, isn't it?\n\n## iOS deep linking\n\nNow let's take a look at the hardest part, which is making this work on iOS. There is no mobile command for iOS so we need to take a look at a basic flow on how to use a deep link with iOS. A deep link can be opened through the terminal with the following command:\n\n```\nxcrun simctl openurl booted theapp://login//\n```\n\nBut this will not be a cross-device solution, especially when you are using a local grid or a cloud solution, and have no access to `simctl`.\n\nDeep links can also be opened from Safari, meaning that if the deep link is entered in the Safari address bar it will trigger a confirmation pop-up with the question of whether the user does indeed want to open the 3rd-party app (see below).\n\n

\n \"Opening\n

\n\n\nWhen the pop-up has been confirmed the app will be opened and the deep link will bring us to the screen we wanted to open. This is like a fairly cross-device solution and manual tests have proven that this works for simulators and real devices.\n\nTo be able to automate this flow with Appium we need to follow some simple steps:\n\n1. Terminate the app under test (optional)\n2. Launch Safari and enter the deep link in the address bar\n3. Confirm the notification pop-up\n\nEach step will be explained in detail below.\n\n### Step 1: Terminate the app\n\n> This is an optional step and only needed if you experience flakiness when not terminating the app or if terminating the app is part of your flow.\n\nIn [Edition 6](https://appiumpro.com/editions/6), Jonathan explained how to test iOS upgrades and he mentioned that as of Appium 1.8.0 we have the [`mobile: terminateApp` command](http://appium.io/docs/en/writing-running-appium/ios/ios-xctest-mobile-apps-management/index.html#mobile-terminateapp). The command needs the `bundleId` of the iOS app as an argument and when that is put together we get this command:\n\n```js\ndriver.execute(\n 'mobile: terminateApp,\n {\n bundleId: \"io.cloudgrey.the-app\"\n }\n);\n```\n\n### Step 2: Launch Safari and enter the deep link in the address bar\n\nAs explained earlier, we now need to open Safari and set the deep link address. It's best to cut this step into 2 parts: opening Safari and then entering the deep link.\n\nOpening an iOS app can be done with [`mobile: launchApp`](http://appium.io/docs/en/writing-running-appium/ios/ios-xctest-mobile-apps-management/index.html#mobile-launchapp). The command needs the `bundleId` of the iOS app, in this case the `bundleId` of Safari, as an argument and when that is put together we get this command.\n\n> (An overview of all bundleIds of the Apple apps can be found [here](https://github.com/joeblau/apple-bundle-identifiers))\n\n```js\ndriver.execute(\n 'mobile: launchApp',\n {\n bundleId: 'com.apple.mobilesafari'\n }\n);\n```\n\nWhen the `launchApp` command has successfully been executed Safari will be opened like this\n\n

\n \"Opening\n

\n\nNow comes the tricky part. This part cost me some headaches because it was the most flaky part of the deep link process, but I finally got it stable, so let's check it out.\nFirst of all, we need to think about the steps a normal user would take to enter a url in Safari, which would be:\n\n1. Click on the address bar\n2. Enter the url\n3. Submit the url\n\nSecondly, keep in mind that we started Appium with _The App_ in the capabilities meaning we are in the native context. For the next steps we need to stay in the native context, so we can use Appium Desktop to explain the steps we need to take.\n\nWhen we start Appium Desktop and open Safari we need to know the selector of the address bar. Since we know that XPATH might be the slowest locator (see [Edition 8](https://appiumpro.com/editions/8)) we want to use a faster locator. Because the address bar has a `name` attribute with the value `URL` you might think we can use the Accessibility ID locator (which we can), but that one will give us back 2 elements: both the `XCUIElementTypeButton` and the `XCUIElementTypeOther` (which will take longer to work with).\n\n

\n \"Safari\n

\n\nIn this case I would advise that we use the iOS predicate string locator to specifically select the url-button element. I would also advise that we use a wait strategy to be sure that the element is visible and ready for interaction. The code would look something like this:\n\n```js\nconst urlButtonSelector = 'type == \\'XCUIElementTypeButton\\' && name CONTAINS \\'URL\\'';\nconst urlButton = $(`-ios predicate string:${ urlButtonSelector }`);\n\n// Wait for the url button to appear and click on it so the text field will appear\n// iOS 13 now has the keyboard open by default because the URL field has focus when opening the Safari browser\nif (!driver.isKeyboardShown()) {\n urlButton.waitForDisplayed(DEFAULT_TIMEOUT);\n urlButton.click();\n}\n```\n\nIf we would now refresh the screen in Appium Desktop we would see that the `XCUIElementTypeButton` element is not there anymore, but changed to a `XCUIElementTypeTextField` element with the same URL `name` attribute. If we would have used the Accessibility ID locator, then this would have been the second point that could cause flakiness. The reason for this is that the switching of elements might not be picked up at the same speed for Appium, making it refer to the already-disappeared `XCUIElementTypeButton` element, and thus causing the script to fail to interact.\n\n

\n \"Safari\n

\n\nTo set the value we're going to use the iOS predicate string locator again. After setting the url we also need to submit the url. This can be done by clicking on the `Go` button on the keyboard, but this can also become flaky if the keyboard doesn't appear. Appium allows you to also use Unicode characters like Enter (defined as Unicode code point `\\uE007`) when using setValue. This means we can set and submit the url with one command:\n\n```js\nconst urlFieldSelector = 'type == \\'XCUIElementTypeTextField\\' && name CONTAINS \\'URL\\'';\nconst urlField = $(`-ios predicate string:${ urlFieldSelector }`);\n\n// Submit the url and add a break\nurlField.setValue('theapp://login/darlene/testing123\\uE007');\n```\n\nWhen the url has been submitted a notification pop-up appears. This brings us to our last step.\n\n> Note: Keep in mind that this script has been made on a English iOS simulator; if you have a different language the selector text (URL) might be different\n\n### Step 3: Confirm the notification pop-up\n\nAfter submitting the url we only need to wait for the notification pop-up to appear and click on the Open button. To keep the locator strategy aligned we are also going to use the iOS predicate string locator here.\n\n

\n \"Safari\n

\n\nWith the wait command the code would look like this:\n\n```js\n// Wait for the notification and accept it\nconst openSelector = 'type == \\'XCUIElementTypeButton\\' && name CONTAINS \\Open\\'';\nconst openButton = $(`-ios predicate string:${ openSelector }`);\n\nopenButton.waitForDisplayed(DEFAULT_TIMEOUT);\nopenButton.click();\n```\n\n> Note: Keep in mind that this script has been made on a English iOS simulator, if you have a different language the selector text (Open) might be different\n\n\n## Making it cross platform\n\nWe've created a deep link script for Android (a one-liner) and for iOS (more complex), so now let's make it a cross platform helper method that can be used for both platforms. If we stitch all code together we can create the following helper:\n\n```js\n/**\n * Create a cross platform solution for opening a deep link\n *\n * @param {string} url\n */\nexport function openDeepLinkUrl(url) {\n const prefix = 'theapp://';\n\n // WebdriverIO provides the `isIOS` property to determine if the running instance\n // is iOS or Android\n if (driver.isIOS) {\n // This one was optional\n driver.execute('mobile: terminateApp', { bundleId: 'io.cloudgrey.the-app' });\n\n // Launch Safari to open the deep link\n driver.execute('mobile: launchApp', { bundleId: 'com.apple.mobilesafari' });\n\n // Add the deep link url in Safari in the `URL`-field\n // This can be 2 different elements, or the button, or the text field\n // Use the predicate string because the accessibility label will return 2 different types\n // of elements making it flaky to use. With predicate string we can be more precise\n const urlButtonSelector = 'type == \\'XCUIElementTypeButton\\' && name CONTAINS \\'URL\\'';\n const urlFieldSelector = 'type == \\'XCUIElementTypeTextField\\' && name CONTAINS \\'URL\\'';\n const urlButton = $(`-ios predicate string:${ urlButtonSelector }`);\n const urlField = $(`-ios predicate string:${ urlFieldSelector }`);\n\n // Wait for the url button to appear and click on it so the text field will appear\n // iOS 13 now has the keyboard open by default because the URL field has focus when opening the Safari browser\n if (!driver.isKeyboardShown()) {\n urlButton.waitForDisplayed(15000);\n urlButton.click();\n }\n\n // Submit the url and add a break\n urlField.setValue(`${ prefix }${ url }\\uE007`);\n\n // Wait for the notification and accept it\n const openSelector = 'type == \\'XCUIElementTypeButton\\' && name CONTAINS \\'Open\\'';\n const openButton = $(`-ios predicate string:${ openSelector }`);\n openButton.waitForDisplayed(15000);\n\n return openButton.click();\n }\n // Life is so much easier =)\n return driver.execute('mobile:deepLink', {\n url: `${ prefix }${ url }`,\n package: \"io.cloudgrey.the_app\",\n });\n}\n```\n\n> Note: Keep in mind that this script has been made on a English iOS simulator, if you have a different language the selector text (URL/Open) might be different\n\nThe helper (which you can also see [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/javascript/Edition084_cross_platform_deep_links.js)) can then be used like this in some test code:\n\n```js\ndescribe('Deep linking', () => {\n it('should be able to login with a deep link', () => {\n //... do something before\n\n openDeepLinkUrl('login/alice/mypassword);\n\n //... do something after\n });\n});\n```\n\nSo now you never need to worry on how to use deep linking for Android and iOS on emulators, simulators and real devices; they all work!\n","metadata":{"lang":"JavaScript","platform":"All Platforms","subPlatform":"All Devices","title":"Reliably Opening Deep Links Across Platforms and Devices","shortDesc":"Deep links are a great strategy for speeding up test execution. However, due to differences between platforms and virtual vs real devices, they can be challenging to implement. It is possible however, by using a mix of cleverness and old-fashioned brute force.","liveAt":"2019-08-28 10:00","canonicalRef":"https://www.headspin.io/blog/reliably-opening-deep-links-across-platforms-and-devices"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/84-reliably-opening-deep-links-across-platforms-and-devices","slug":"84-reliably-opening-deep-links-across-platforms-and-devices","path":"/editions/84-reliably-opening-deep-links-across-platforms-and-devices","news":null,"tags":["JavaScript","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0079.md","num":79,"rawMd":"\n\nKotlin is a new programming language, which is many ways is the successor to Java. The compiled code runs on the JVM, it is supported by Java IDEs, and its syntax is very similar to Java. It is the officially preferred language for Android app development, and a friend of mine is even rewriting his team's backend API in Kotlin.\n\nKotlin has many features which clean up some of the longstanding issues with Java, and also just gives a fresh start, without needing to support legacy decisions and APIs.\n\nWhat many readers may not know, is that Kotlin code can be easily integrated into a Java project. A Java project can have some code files in Kotlin, a Kotlin program can import Java libraries, and with a little extra work, Java programs can import Kotlin libraries. This allows for a gradual transition, rather than the risks of a complete rewrite.\n\nIn this article, we'll take a simple Appium Java test, and convert it to Kotlin in just a few minutes!\n\nIn my specific case, I started with [our code sample from two weeks ago](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition077_Tuning_WDA_Startup.java). I'm using IntelliJ IDEA, and our project was already set up to use Gradle.\nThe only difficulty I ran into is that the IDE's Kotlin plugins only work with gradle version 4.1 and up. Our project was using gradle version 4.0, so I had to update first, by running the following code:\n\n```bash\n./gradlew wrapper --gradle-version 4.10.3\n```\n\nI then added a new file next to our existing tests, but gave it a name ending in `.kt` (for Kotlin) instead of `.java`.\n\nAs soon as I had finished renaming the file, the IDE displayed a banner prompting me to configure Kotlin. When I clicked on it, the IDE automatically added some boilerplate to the `build.gradle` file to add Kotlin support.\n\nI pasted the following \"hello world\" code into the file, and then restarted the IDE, clearing all caches.\n\n```kotlin\nfun main() {\n println(\"hello world\")\n}\n```\n\nAs soon as the IDE came back online, this code was parsed successfully and a green \"play button\" arrow was displayed next to the `main` function defined in the code. Clicking this icon to run, the words \"hello world\" were printed to the console.\n\n

\n \"Breakdown\n

\n\nGreat! We have Kotlin code running in our project, now let's demonstrate a simple JUnit test.\n\nAll it takes is importing JUnit, adding the usual `@Test` annotation to our method, and putting our method into a class.\n\n```kotlin\nimport org.junit.Assert.assertEquals\nimport org.junit.Test\n\nclass Tests {\n @Test\n fun mainTest() {\n assertEquals(\"hello\", \"hello\")\n }\n}\n```\n\nNow, running the code by clicking on the green play button icons, runs the tests and output is displayed the way unit tests are normally viewed in the IDE.\n\n

\n \"Breakdown\n

\n\nHaving demonstrated to ourselves that we can run a basic test, let's convert our Appium test from Java to Kotlin. This is a chance to highlight a built-in feature which really amazed me.\n\nWe go into the Java test file, highlight all the code, __COPY__. Now in our Kotlin file, __PASTE__:\n\n

\n \"Breakdown\n

\n\nAnd 95% of the work is done for us, leaving us with perfectly readable code.\n\nThe test file won't actually compile though. We picked a good example that brought up a case that involves some manual intervention. The `firstTest` static variable from our original class has been converted to a \"[companion object](https://kotlinlang.org/docs/tutorials/kotlin-for-py/objects-and-companion-objects.html)\" which is how Kotlin implements static class members. Kotlin is a different language after all.\n\nThe compiler complains about our use of `firstTest`. This is related to the most obvious difference between Java and Kotlin at first glance, and also my favorite feature so far. Kotlin uses a concept called \"nullable types\", where a variable of any class _cannot_ be null. In order to have a null value, the variable type must be set to the \"nullable\" version of the type, indicated by following the class name with a `?`. So a `String` variable can never be null, while a `String?` variable can be null or a String. This applies for every class. If you define a method which takes a `String` parameter, you never need to check if it is null before you use it, because _it's guaranteed to not be null_.\n\nThere's a lot more to this concept, including some interesting syntax and shortcuts for null-checking, and there are plenty of articles explaining it in detail online, as well as the [Kotlin documentation](https://kotlinlang.org/docs/reference/null-safety.html). This concept has existed for a long while, and is able to prevent many null reference exceptions and other issues. Both Swift and Kotlin implement it.\n\nOur copy-pasted code fails because we are attempting to use the not operator `!` on a variable of type `Boolean?` and the compiler is complaining that the `Boolean?` could be null, and it isn't able to calculate the results of `!null`. Since the code sets a value for `isFirst` at the declaration site, we can convert it to the non-nullable type `Boolean`, and our code now compiles.\n\nAnd tests pass!\n\nThe Java Appium client worked from the Kotlin code with no problem.\n\nOur test was pretty simple, but I hope you consider learning Kotlin and experiment with converting your code. As time moves on, we will see if Kotlin adoption grows, but hopefully this will allay your fears about migrating. It can be done easily and one file at a time.\n\nBelow is the full Kotlin test file, which can also be found in our [repository of examples](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition079_Converting_A_Test_To_Kotlin.kt).\n\n```kotlin\nimport org.junit.Test\n\nimport io.appium.java_client.MobileElement\nimport io.appium.java_client.ios.IOSDriver\nimport org.junit.After\nimport org.junit.Before\nimport org.openqa.selenium.remote.DesiredCapabilities\n\nimport java.io.IOException\nimport java.net.URL\n\nimport junit.framework.TestCase.assertEquals\n\n// Converted test \"Edition077_Tuning_WDA_Startup\" from Java to Kotlin just by copy-pasting.\n// the `firstTest` variable was manually changed from type Boolean? to Boolean, in order to compile.\n// A few comments have been cleaned up, just to make this example easier to use and read.\nclass Edition078_Converting_A_Test_To_Kotlin {\n private val APP : String = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip\";\n\n private var driver: IOSDriver<*>? = null\n\n @Before\n @Throws(IOException::class)\n fun setUp() {\n val caps = DesiredCapabilities()\n caps.setCapability(\"platformName\", \"iOS\")\n caps.setCapability(\"platformVersion\", \"12.2\")\n caps.setCapability(\"deviceName\", \"iPhone Xs\")\n caps.setCapability(\"automationName\", \"XCUITest\")\n\n caps.setCapability(\"app\", APP)\n\n caps.setCapability(\"noReset\", true)\n caps.setCapability(\"udid\", \"009D8025-28AB-4A1B-A7C8-85A9F6FDBE95\")\n caps.setCapability(\"bundleId\", \"io.cloudgrey.the-app\")\n\n if ((!firstTest)) {\n caps.setCapability(\"webDriverAgentUrl\", \"http://localhost:8100\")\n }\n\n driver = IOSDriver(URL(\"http://localhost:4723/wd/hub\"), caps)\n }\n\n @After\n fun tearDown() {\n firstTest = false\n try {\n driver!!.quit()\n } catch (ign: Exception) {\n }\n\n }\n\n @Test\n fun testA() {\n assertEquals(1, 1)\n }\n\n @Test\n fun testB() {\n assertEquals(1, 1)\n }\n\n @Test\n fun testC() {\n assertEquals(1, 1)\n }\n\n @Test\n fun testD() {\n assertEquals(1, 1)\n }\n\n @Test\n fun testE() {\n assertEquals(1, 1)\n }\n\n @Test\n fun testF() {\n assertEquals(1, 1)\n }\n\n companion object {\n private var firstTest: Boolean? = true\n }\n}\n```\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Converting Java Tests to Kotlin","shortDesc":"Kotlin is a growing language which is very compatible with Java. See how simply a Java Appium test can be converted to Kotlin","liveAt":"2019-07-24 10:00","canonicalRef":"https://www.headspin.io/blog/converting-java-tests-to-kotlin"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/79-converting-java-tests-to-kotlin","slug":"79-converting-java-tests-to-kotlin","path":"/editions/79-converting-java-tests-to-kotlin","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0078.md","num":78,"rawMd":"\n\nEvery Appium automation session is given a unique ID when the session is first created. Webdriver HTTP commands after the initial POST to the `/session` endpoint all have paths which start with `/session/:sessionId`, where `:sessionId` is the unique identifier for this session, returned in the initial `/session` response.\n\nSince Appium has a one-to-one relationship between sessions and devices, if a session is already running, any new sessions will automate other devices available to the Appium server. A device will not be reused for a session until the session is ended via an HTTP DELETE request sent to `/session/:sessionId`.\n\nWhen it comes to your test script, the only thing which differentiates the client (sometimes called `driver`) object from the client for a different test, is the session ID. All the client libraries automatically save the session ID when a sessions is started and include in in the path for all further commands. This is the only state a client object needs to keep, and we can take advantage of this to unlock some useful flexibility.\n\nI, and others, have run into many cases where we want to control an automation session which is already in progress, but cannot share the client object itself. Sometimes I will have a Java test script paused halfway through a test in debugging mode, and want to connect my Javascript REPL to quickly try out some commands in the current app state. I can't pass the Java client object to my Javascript REPL, but I can easily copy and paste the session ID into my REPL and connect the client.\n\nAnother use case testers run into is when, for some reason driven by necessity, part of a test run is operated in a different scope. It could be you want to duplicate the client object and pass it to another thread, or maybe you are recovering from an error where the driver object has been destroyed but you need to delete the session on the server still. Maybe you are in a very unfortunate position where your test codebase is split across projects and you need to continue your session in another language entirely.\n\nThis is a pattern which should not be used in an average test suite, but I've seen many cases where it's use is very valid.\n\nFor most clients this trick is very easy. The clients have a method for getting the current session ID, and another accessible field for setting the session ID. There are examples of how to do this online for most of the clients, but the Java solution is particularly difficult. Years ago it was relatively easy, but recent updates have made it trying.\n\nI'm working on a [pull request](https://github.com/appium/java-client/pull/1181) to the Appium Java client to add a new constructor which make this very easy. I wasn't able to finish in time for this article, but next week the following constructor should be available for all to use:\n\n```java\nString sessionId = \"d8080676-634d-49fc-8624-fc7b57c5d530\";\nAppiumDriver driver = new AppiumDriver(\"http://localhost:4723/wd/hub\" , sessionId);\n```\n\nThis results in a client object which will send commands to the given session ID, without starting a new session. Notice we don't need desired capabilities, since the session has already been started. If you need to get the desired capabilities which created the session, they are returned by the `getAllSessionDetails()` method.\n\nThis was much trickier to implement than I bargained for: the Selenium client which the Appium client extends attempts to create a new session as part of the constructor. Since we want to connect to an existing session, we have to skip this initialization. I ended up taking a slightly different approach, but could have saved myself some frustration if I had read this [great explanation of the code involved](https://tarunlalwani.com/post/reusing-existing-browser-session-selenium-java/) posted by [Tarun Lalwani](https://tarunlalwani.com/).\n\nWe will attempt to introduce these changes in the Selenium client so even more can benefit. Hopefully this saves some headaches.\n\nThat's all for this week. Once the pull request is approved and a new version published, this [sample code](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition078_Attaching_To_Existing_Session.java) will work.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Attaching Appium Clients to Existing Sessions","shortDesc":"Attaching an Appium client to an existing session can come in handy, and now it's even easier with a new constructor for the Java client.","liveAt":"2019-07-17 10:00","canonicalRef":"https://headspin-test.webflow.io/blog/attaching-appium-clients-to-existing-sessions"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/78-attaching-appium-clients-to-existing-sessions","slug":"78-attaching-appium-clients-to-existing-sessions","path":"/editions/78-attaching-appium-clients-to-existing-sessions","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0077.md","num":77,"rawMd":"\n\nSome Appium users have asked me how to speed up their iOS tests, citing the length of time it takes to start tests which use the WebDriverAgent library (all tests using the XCUITest driver).\n\nMost of the perceived speed of an Appium test can't be improved due to the base speed of booting simulators or the UI actions themselves. The slowest part, which users were asking me how to avoid, is the initial startup of a test: the time between sending the first `POST /session` command and the response indicating that your test script can begin sending commands. We'll call this time period the \"session creation\" time.\n\nLet's start with the most basic iOS test:\n\n```java\nprivate String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip\";\n\n@Before\npublic void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.2\");\n caps.setCapability(\"deviceName\", \"iPhone Xs\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n\n caps.setCapability(\"app\", APP);\n\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n}\n\n@After\npublic void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n}\n\n@Test\npublic void testA() {\n assertEquals(1,1);\n}\n\n@Test\npublic void testB() {\n assertEquals(1,1);\n}\n\n@Test\npublic void testC() {\n assertEquals(1,1);\n}\n\n@Test\npublic void testD() {\n assertEquals(1,1);\n}\n\n@Test\npublic void testE() {\n assertEquals(1,1);\n}\n\n@Test\npublic void testF() {\n assertEquals(1,1);\n}\n```\n\nThis is basically the default template we use for an iOS test in the [AppiumPro sample code repository](https://github.com/cloudgrey-io/appiumpro). We have our sample app hosted on a web Url for convenience, a `@Before` step which creates a fresh session for each test, an `@After` step to delete the session at the end of each test, followed by six tests which do nothing.\n\nEach of the six tests take __12.8__ seconds on average. We can cut this down by two thirds!\n\nThere are desired capabilities we can specify to greatly reduce the time it takes to create a session. Appium is built to cater to a large number of devices, for use in many different situations, but we also want to make sure that it is easy to get started automating your first test. When specifying desired capabilities, Appium will analyze the state of your system and choose default values for every desired capability which you don't specify. By being more specific, we can have Appium skip the work it does to choose the default values.\n\nOur first improvement is to set the `app` location to a file already on the host device. Downloading an app from a remote source is adding __3.9__ seconds to each test in the suite.\n\n```java\nprivate String APP = \"/Users/jonahss/Workspace/TheApp-v1.9.0.app.zip\";\n```\n\nYour test suite probably already does things this way, but for our demo repository this really speeds things up.\n\nRunning the tests, it's easy to notice that the app gets reinstalled on the simulator for each test. This takes a lot of time, and can be skipped. You may have certain tests which require a fresh install, or need all the app data cleaned, but those tests could be put into a separate suite, leaving the majority of tests to run faster by reusing the same app. Most users should be familiar with the `noReset` desired capability.\n\n```java\ncaps.setCapability(\"noReset\", true);\n```\n\nThis saves an additional __2.9__ seconds per test.\n\nThat was the easy stuff, giving us an average startup time of __6__ seconds per test, but we can shave off another __2.1__ seconds, which is a 35% improvement.\n\nAppium uses the `simctl` commandline tool provided by Apple to match the `deviceName` desired capability to the udid of the simulator. We can skip this step by specifying the simulator udid ourselves. I looked through the logs of the previously run test and took the udid from there:\n\n```java\ncaps.setCapability(\"udid\", \"009D8025-28AB-4A1B-A7C8-85A9F6FDBE95\");\n```\n\nThis saves __0.4__ seconds per test.\n\nAppium also gets the bundle ID for your app by parsing the app's plist. We can supply the known bundle ID, again taking it from the logs of the last successful test. This ends up saving us another __0.1__ seconds:\n\n```java\ncaps.setCapability(\"bundleId\", \"io.cloudgrey.the-app\");\n```\n\nWhen loading the WedDriverAgent server, Appium loads the files from wherever XCode saved it after compilation. This location is called the \"Derived Data Directory\" and Appium executes an `xcodebuild` command in order to get the location. Again, we can look through the logs of the last test run and supply this value ourselves, allowing Appium to skip the work of calculating it:\n\n```java\ncaps.setCapability(\"derivedDataPath\", \"/Users/jonahss/Library/Developer/Xcode/DerivedData/WebDriverAgent-apridxpigtzdjdecthgzpygcmdkp\");\n```\n\nThis saves a whopping __1.4__ seconds. While `xcodebuild` is useful, it hasn't been optimized to supply this bit of information.\n\nThe last optimization is to specify the `webDriverAgentUrl` desired capability. If specified, Appium skips a step where it checks to make sure that there are no obsolete or abandoned WebDriverAgent processes still running. The WebDriverAgent server needs to already be running at this location, so we can only use this desired capability after the first test starts the server.\n\n```java\ncaps.setCapability(\"webDriverAgentUrl\", \"http://localhost:8100\");\n```\n\nSo what have we done? Using a local file and `noReset` reduced the base test time from __12.8__ seconds to __6__ seconds, making test startup __53.3%__ faster.\n\nOn top of that, we performed some more obscure optimizations to improve test times another __23.9%__, shaving off another __1.3__ seconds.\n\nHere's a visual breakdown of what Appium spends its time doing during one of our tests:\n\n

\n \"Breakdown\n

\n\n\"Remaining test time\" and \"session shutdown\" are all that remain in our optimized test:\n\n```java\nprivate String APP = \"/Users/jonahss/Workspace/TheApp-v1.9.0.app.zip\";\n\nprivate IOSDriver driver;\nprivate static Boolean firstTest = true;\n\n@Before\npublic void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.2\");\n caps.setCapability(\"deviceName\", \"iPhone Xs\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n\n caps.setCapability(\"app\", APP);\n\n caps.setCapability(\"noReset\", true);\n caps.setCapability(\"udid\", \"009D8025-28AB-4A1B-A7C8-85A9F6FDBE95\");\n caps.setCapability(\"bundleId\", \"io.cloudgrey.the-app\");\n caps.setCapability(\"derivedDataPath\", \"/Users/jonahss/Library/Developer/Xcode/DerivedData/WebDriverAgent-apridxpigtzdjdecthgzpygcmdkp\");\n if (!firstTest) {\n caps.setCapability(\"webDriverAgentUrl\", \"http://localhost:8100\");\n }\n\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n}\n\n@After\npublic void tearDown() {\n firstTest = false;\n try {\n driver.quit();\n } catch (Exception ign) {}\n}\n\n@Test\npublic void testA() {\n assertEquals(1,1);\n}\n\n//... 5 more tests\n```\n\nFor all these tests, I kept the same simulator running. Starting up a simulator for the first test takes some extra time. I also noticed that the first test of every suite always takes a little bit longer even when the simulator is already running. For the timing statistics in this article, I omitted the first test from each run. None of them were that much longer than the average test, and the goal is to reduce total test suite time. If you have 100 tests, an extra second on the first one doesn't impact the total suite time much.\n\nThat said, all this is splitting hairs, since as soon as your tests contain more than a few commands, the amount of time spent on startup is insignificant compared to the amount of time spent finding elements, waiting for animations, and loading assets in your app.\n\nFull example code located [here](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition077_Tuning_WDA_Startup.java).\n\nFor situations where a CI server is running Appium tests for the first time, further optimizations can be made to reduce the time spent compiling WebDriverAgent for the first test. We'll discuss these in a future edition of Appium Pro!\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Optimizing WebDriverAgent Startup Performance","shortDesc":"Our basic demo test suite can be optimized by 67%, reducing the length of every test from 12.8 seconds to 4.2 seconds.","liveAt":"2019-07-10 10:00","canonicalRef":"https://www.headspin.io/blog/optimizing-webdriveragent-startup-performance"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/77-optimizing-webdriveragent-startup-performance","slug":"77-optimizing-webdriveragent-startup-performance","path":"/editions/77-optimizing-webdriveragent-startup-performance","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0076.md","num":76,"rawMd":"\n\n> This article is based on a [webinar](https://register.gotowebinar.com/recording/4229598430933236231) that was hosted by [HeadSpin](https://ui.headspin.io/register?referral=start-testing-appiumpro) in June 2019. Be sure to check out the recording as well!\n\n

\n \"React\n

\n\n## Intro to React Native\n\n[React Native](https://facebook.github.io/react-native/) is a response to the age-old problem of how to get an agreeable cross-platform mobile app developer experience while not sacrificing _user_ experience or perceived performance. Developers love building apps with a single set of tools and methodologies, and this of course means _web_ development tools and methodologies, since the web is the canonical universal platform. Unfortunately, building native apps on different mobile platforms (Android and iOS) often involves using a host of _different_ tools and methodologies, based on whatever Google or Apple dictate to their developer communities.\n\nChoosing to build a hybrid app was always a way out of the double-platform problem, but hybrid apps came with lots of perceived UX issues. This is where React Native shines: rather than writing web-like code which actually runs in a webview, you use React Native to write web-like code which is interpreted on the fly inside of a _native_ mobile app, which creates _native_ UI components and response, and which responds to _native_ UI events. The end-user experience is (at least in principle) exactly the same as any native mobile app built with the first-party platform SDKs, but the developer experience is that of a web developer, writing in React-flavoured JavaScript, and using JSX along with a CSS equivalent for structure and styling.\n\nHow does React Native accomplish this best of both worlds feat? Between the app code and the user experience it interposes several unique modules, as depicted in the following diagram:\n\n

\n \"React\n

\n\nBasically, when a React Native app is launched, regular old native app code is run. You don't write this code---it's maintained as part of the React Native project. In addition to these native (per-platform) modules, React Native apps come bundled with a JavaScript engine called [JavaScript Core](https://github.com/WebKit/webkit/tree/master/Source/JavaScriptCore). The code you write is executed by this JS engine, and any time this results in a UI update, a message is passed to the native modules via a special message bus called the React Native Bridge. Once the native modules again have control, they are free to update the UI, create native UI components, or listen for user input, just as any native app would.\n\nThis multi-part architecture certainly adds internal complexity to the app, but its complexity which is hidden away by the architecture. A React Native app developer can usually ignore it, and just live in the world of React.js, all the while building apps which produce native user experiences. Well, mostly. At the end of the day, these apps still need to be compiled as iOS `.ipa` files or Android `.apk` files. They still need to have the appropriate app manifests and resources in the appropriate places. So there is some amount of platform-specific wrangling that still needs to be done, even with React Native. How much exactly depends on your specific app, and how differently-designed you _want_ your apps to be. But in general, a more accurate picture of the architecture, including your responsibilities, is as pictured here:\n\n

\n \"React\n

\n\nBased on my experience maintaining a React Native app ([The App](https://github.com/cloudgrey-io/the-app), which is used for almost all of Appium Pro's examples), these are the pros and cons of using React Native for mobile app development:\n\n|Pros|Cons|\n|----|----|\n|Mostly single codebase|A lot of \"mosts\"—need some iOS/Android experience|\n|Mostly no need to learn iOS or Android app dev|Reliant on RN team for SDK compatibility updates|\n|Good debugging toolkit for rapid development|Difficult to write truly platform-specific UX|\n|Mostly native app speed / UX|Reliant on project owned by FB (though now under MIT license)|\n|Extensible—write your own native modules| |\n\n## Testing React Native Apps\n\nAs with any kind of app, there are different levels of testing to consider, from unit testing up to end-to-end or UI testing. The most popular option for unit testing React Native apps is [Jest](https://jestjs.io/docs/en/tutorial-react-native.html), produced by Facebook (same as React Native). This kind of testing doesn't involve UI components at all. Jest can be combined with another library (like [react-native-testing-library](https://github.com/callstack/react-native-testing-library)), to get a little more fancy and test at the component level (though again, this is not testing involving native UI components, only virtual components).\n\nWhen it comes to UI testing, Appium is a great option for testing React Native apps. React Native apps just _are_ native apps, therefore they're testable out of the box with Appium. React Native components even respond to a [`testID`](https://facebook.github.io/react-native/docs/view#testid) attribute that allows developers to attach test-friendly labels to components. However, there are a few things to consider when developing React Native apps to make sure that they are _fully_ testable with Appium. In the Appium Desktop screenshot below, for example, you will notice a pretty substantial problem with this React Native app:\n\n

\n \"No\n

\n\nThe problem is that, even though I can clearly see many elements with lots of unique text labels and so on, all I can find in the XML hierarchy is a single `android.widget.FrameLayout`! The reason for this is that Appium's element finding algorithms operate on the UI Accessibility layer, and the designer of this React Native app has designated certain elements as important for accessibility, thus rendering many _other_ elements _un_-important for accessibility. Oops!\n\nThe solution here is to make sure not just to use the `testID` attribute on important components, but also to set the [`accessibilityLabel`](https://facebook.github.io/react-native/docs/view#accessibilitylabel) attribute, to ensure that the element is always findable via Appium's 'accessibility id' locator strategy. (Actually, this is only a problem on Android, because on Android, React Native puts the `testID` into something called a 'view tag', which is not accessible to Appium at all.\n\nI like to make a convenience function called `testProps` to make this process easy:\n\n```js\nexport function testProps (id) {\n return {testID: id, accessibilityLabel: id};\n}\n```\n\nNow I can just use this function everywhere instead of `testID`:\n\n```jsx\n\n```\n\nAnother tactic for dealing with this issue, if we don't want to add labels and IDs to lots of components, is to make sure that parent components have their `accessible` attribute set to `false`; this allows the accessibility visibility of children to bubble up through a parent, rather than treating the parent component as a monolithic component from the perspective of the accessibility system.\n\nThat's really it! I haven't run into any other significant issues testing React Native apps with Appium. But check out the [webinar](https://register.gotowebinar.com/recording/4229598430933236231) for a more in-depth discussion, and a comparison of Appium with the Detox UI testing framework, which has some special integrations with React Native specifically.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Testing React Native Apps with Appium","shortDesc":"React Native is a popular app development framework that enables native user experiences while maintaining a mostly single platform developer experience. Testing React Native apps with Appium is relatively straightforward, but there are a few tips that might make the difference between a successful and and an unsuccessful Appium testing expedition.","liveAt":"2019-07-03 10:00","canonicalRef":"https://www.headspin.io/blog/testing-react-native-apps-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/76-testing-react-native-apps-with-appium","slug":"76-testing-react-native-apps-with-appium","path":"/editions/76-testing-react-native-apps-with-appium","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0075.md","num":75,"rawMd":"\n\n> This article is the second in a two-part series on using Appium with platforms _very_ different from mobile operating systems. Make sure to read the [first part](https://appiumpro.com/editions/74) first!\n\n

\n \"Custom-built\n

\n\nWe ended the first part of this series with a fully functional drum machine, powered by a Circuit Playground Express and some simple hardware components like buttons and LEDs. Now we come to the question of how to _test_ this device. As we begin to think about the question, we realize that there are several different places we can insert a test component in the application \"stack\". For reference, here's the app \"stack\":\n\n

\n \"Stack\n

\n\nNow, any good test must (1) trigger, synthesize, or simulate some inputs, and (2) validate the outputs. For the purposes of this exploration, we're going to focus on the first half of this--the input automation. (Verifying the output in this case would involve capturing the audio and running it through some kind of [audio fingerprint analysis](https://appiumpro.com/editions/70), and setting up a camera or light sensor to track the LEDs.) So just considering the input, we have 3 possible places to hook into the stack with our automation:\n\n

\n \"Stack\n

\n\n1. We could build a robot to press the buttons\n2. We could create a device to send electrical signals to the I/O pins that the buttons are connected to\n3. We could write code that triggers the same routines in the processor that are triggered by a button press\n\nSo which level of automation should we choose? It really depends on what we care most about testing, and what we have the ability to do. By far the easiest thing to do would be to write code that simply pretends a button has been pressed. But this is pretty unsatisfying, because by doing this we're not actually testing the _device_, only the microcontroller on to the outputs.\n\nUltimately, I choose level #2, mostly because I don't know how to build robots that tap buttons on command. And I felt #2 is still a reasonable \"end-to-end\" test of the drum machine. We're not testing the buttons, true, but we can probably assume they've been well-tested by the button manufacturer. So what we want is a way to trigger electrical signals to the CPX's I/O pins in exactly the same way that a button press would.\n\nWell, the best way to trigger electrical signals on command is via... another microcontroller! Enter the [Raspberry Pi 3 Model B](https://www.raspberrypi.org/products/raspberry-pi-3-model-b/). This is a little computer that happens to have a set of its own [GPIO pins](https://www.raspberrypi.org/documentation/usage/gpio/). What this means is that we can program a Raspberry Pi to send whatever electrical signals we want, including ones that emulate what happens when the buttons of our drum machine are pressed. This is what our Raspberry Pi looks like with wires attached to the GPIO pins:\n\n

\n \"Raspberry\n

\n\nNow all we need is a way to automate the Raspberry Pi GPIO pins, and that is where we can involve Appium, because there is now an [Appium driver for the Raspberry Pi](https://github.com/jlipps/appium-raspi-driver). I created this driver with this application in mind, and so it is very limited in its functionality. All it can do is start and stop sessions, and control whether a particular pin is turned on or off. Luckily, that's all we need. Diagrammatically, it looks something like this:\n\n

\n \"Raspberry\n

\n\n(If you're interested in seeing how the Appium Raspberry Pi driver works under the hood, have a look at its source code. It extends Appium's BaseDriver class, and mixes in functionality from the Johnny Five Node.js library, which enables easy control of the board).\n\nBecause the Raspberry Pi is a full-on computer that can run Node.js, we can simply run the Appium driver on it directly (after cloning the repo and running `npm install` in it, of course, and ensuring our Pi has all the requisite dependencies):\n\n```\ngit clone https://github.com/jlipps/appium-raspi-driver.git\ncd appium-raspi-driver\nnpm install\n\n# now start the driver in standalone mode\nnode . -a raspberrypi.local\n```\n\nNotice that we have to be explicit about the address we want our server to run on, if we want the server to be accessible from a computer running the test script elsewhere on the network (or we could just run the test script on the Pi).\n\nSo that takes care of the server, but we still have to worry about writing a test to execute on it. As with any Appium test, there are two basic components to a test using this driver: the capabilities, and the test steps themselves. For the Appium Raspberry Pi driver running in standalone mode, all we need is one capability: `app`. This capability is very important, because it tells the driver which pins we want to automate, and what their initial states will be. If you think about it, we need to have 3 pieces of information in order to automate this scenario successfully:\n\n1. Which pin(s) on the Raspberry Pi are we using to control our device-under-test?\n2. Which pin(s) on the device-under-test are we targeting for automation?\n3. What should the initial state of the Raspberry Pi pins be? (I.e., should they be in input or output mode, and set to high or low?)\n\nThe reason we care about #2 is that, as we develop our test, we would prefer to work with \"elements\" whose names correspond to the appropriate places on the device-under-test, _not_ the Raspberry Pi pins themselves (because what if we choose to plug them into different pins next time we run the test? That would make our locators 'brittle'). Ultimately, the `app` capability I defined looked like this:\n\n```js\n{\n pins: {\n 'P1-7': { id: 'A1', mode: 'output', init: 1, },\n 'P1-11': { id: 'A2', mode: 'output', init: 1, },\n 'P1-13': { id: 'A5', mode: 'output', init: 1, },\n 'P1-15': { id: 'A6', mode: 'output', init: 1, },\n }\n}\n```\n\nIn other words, it specifies a set of 4 pin mappings. The name of each pin is its name on the Rasbperry Pi GPIO header. The associated `id` is the name of the terminal on the CPX that we want to send (or stop sending) electrical signals to. The `mode` of the pin is either input or output, and `init` can be either `1` or `0` based on whether we want the initial state of the pin at the beginning of the test to be high or low.\n\nWhat this `app` definition allows us to do is work with \"elements\" in our test code, just as if we were looking for UI elements on a screen:\n\n```\nconst kick = await driver.elementById(\"A1\");\n```\n\nIn this example, we are finding an \"element\" named \"A1\". Since we have provided the Raspberry Pi driver with our app config, the driver knows that when we interact with this element, we will be sending signals via the \"P1-7\" output pin. And of course, we could have named \"A1\" anything we want, even something more human-readable like \"KickDrum\"--I chose to name it after the names of the IO pins on the CPX, so I would always know what wires to connect where. (As testers, if we had access to the app code, we could of course auto-generate this app definition to ensure everything works no matter where we've chosen to connect our wires).\n\nOK, so we can find an element. But what can we do with it? The only thing we can do with elements retrieved via this driver is set them to a digital high or low signal, represented by the strings `\"1\"` and `\"0\"` respectively. To register these values, we use the plain old `sendKeys` command!\n\n```js\nawait kick.sendKeys(\"0\")\n```\n\nThe above command tells the pin represented by the previously-retrieved element to change to a low/zero signal.\n\nThat is all that is possible with the current state of the Appium Raspberry Pi driver, though in the future it would be easy to add support for higher-level components available on the Pi via the Johnny Five API---but for the sake of simply driving another electronic device via signals in this way, the current set of commands is all we need.\n\nSo all that remains for us is to actually write some test code! Here's the code I used in my [AppiumConf 2019 talk demo](https://www.youtube.com/watch?v=zWXQ_TRYKic). It's written in JavaScript, because that was the language that I felt most comfortable developing the non-Appium portions of the code is (the code responsible for parsing a musical notation string into WebDriver commands). But like any Appium script, this could have been written in any language. First of all, let's look at the main function:\n\n```js\n(async function main () {\n driver = wd('http://raspberrypi.local:7774/wd/hub');\n await driver.init({app});\n await B.delay(2000);\n try {\n const kick = await driver.elementById(\"A1\");\n const snare = await driver.elementById(\"A2\");\n const hat = await driver.elementById(\"A5\");\n const tom = await driver.elementById(\"A6\");\n const elMap = {k: kick, s: snare, h: hat, t: tom};\n const song = `\n k4 k4 s8 k4 k8\n k4 k4 s8 k8 s8 s8\n h4 k4 s8 k4 k8\n k4 k4 s8 k8 t8 t8\n h4\n `;\n await playSong(song, elMap);\n } finally {\n await driver.quit();\n }\n})().catch(console.error);\n```\n\nYou can immediately see that this isn't a proper test, just an automation script (there's no test method or assertions, just plain old JS functions). We first connect to the Appium Raspberry Pi driver running on the Pi at its default port of 4774 (this is where it was important to start the driver with a non-default address, so that connecting to it over Ethernet works). Then, we pass in our app definition (as shown above). Finally, we assign names to our different pin elements, and then call a custom `playSong` method which directs the client to send the right signals at the right time based on what is defined in `song`.\n\nThe implementation of `playSong` isn't important to show here, but what is important is that somewhere inside that method are a bunch of calls to another method `hit`, which does the actual work:\n\n```js\nasync function hit (el, delay=HIT_HOLD_MS) {\n await el.sendKeys(\"0\");\n await B.delay(delay);\n await el.sendKeys(\"1\");\n}\n```\n\nYou can see here that there's really no magic involved: we simply turn a pin's signal off and on in order to emulate the act of pressing a button on a device. (If you're confused, like I was, why we have to turn the signal _off_ in order to trigger a drum hit, it appears to be because that's how the button is designed--when it is pressed down, it _breaks_ the circuit, so that is how I programmed the CPX to recognize a drum hit. Therefore we have to follow the same pattern with our test logic, and that is why the initial state of all the pins was set to `1`).\n\nThere you go! As I said above, this is really only half of a true test, since there is no verification involved, but it does show how, with a little bit of imagination and creativity, the Appium framework can be repurposed for applications _very_ different from traditional screen-based UI apps. The Appium Raspberry Pi driver is available for you to use however you wish. In fact, the Raspberry Pi is probably overkill for the purpose of simply driving electrical signals to a device under test. Something like one of the smaller Arduinos would probably be more appropriate---I just happened to have a Pi on hand!\n\nIf you want to see my full test script in the context of a whole project, you can find it [on GitHub](https://github.com/jlipps/appiumconf2019/tree/master/raspi). Please let me know if you find any interesting uses for this approach, or end up doing something similar for your own IoT devices!\n","metadata":{"lang":"JavaScript","platform":"Raspberry Pi","subPlatform":"All Devices","title":"Automating Custom IoT Devices With Appium, Part 2","shortDesc":"Appium can go far beyond automation of mobile apps. In this second installment of the Appium + IoT series, we look at how to write test code that runs on a custom Appium driver built for the Raspberry Pi GPIO pins.","liveAt":"2019-06-26 10:00","canonicalRef":"https://www.headspin.io/blog/automating-custom-iot-devices-with-appium-part-2"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/75-automating-custom-iot-devices-with-appium-part-2","slug":"75-automating-custom-iot-devices-with-appium-part-2","path":"/editions/75-automating-custom-iot-devices-with-appium-part-2","news":{"rawMd":"I'm happy to announce that yesterday a brand new Appium e-learning course was released, called [Mobile Testing With Appium](https://www.linkedin.com/learning/mobile-testing-with-appium/) (also available at [Lynda.com](https://www.lynda.com/Selenium-tutorials/Mobile-Testing-Appium/5030986-2.html)). I produced it in conjunction with LinkedIn Learning, and the result is a very high quality introductory course that takes you through install, setup, and the basic skills you need to learn in order successfully automate mobile native apps. You may already be an Appium Pro, but please forward it on to anyone you know who's trying to get started! And when you've gone as far as you can with this course, don't forget that [Cloud Grey](https://cloudgrey.io) offers custom multi-day Appium training workshops for enterprise teams, to take you the rest of the way!\n","num":75,"mdPath":"/vercel/path0/content/news/0075.md"},"tags":["JavaScript","Raspberry Pi","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0074.md","num":74,"rawMd":"\n\n> This article is the first in a two-part series on using Appium with platforms _very_ different from mobile operating systems. You can also read [part two](https://appiumpro.com/editions/75-automating-custom-iot-devices-with-appium-part-2).\n\n

\n \"Custom-built\n

\n\nIn my [AppiumConf 2019 opening keynote](https://www.youtube.com/watch?v=zWXQ_TRYKic), I made some appropriately broad and keynotey comments about the state of automated test problems and techniques. But really, this was all just setup, designed to demonstrate a single point: Appium is first and foremost a set of code interfaces that make it easy to bring Webdriver-style automation to _any_ platform.\n\nOver the years, I've been thinking a bit about what odd platforms are out there that we could automate with Appium, and one that's kept coming to mind is \"IoT\". IoT of course is a vague term, and can refer to lots of types of applications. One constant for definitions of IoT is that it involves some kind of physical connection to the world. IoT devices aren't _just_ software---they're a combination of software _and_ some kind of special-purpose hardware (usually connected to the Internet, as the term implies). This hardware has interface points with software, but crucial to its existence is some kind of physical input and/or output.\n\nHow on earth would we test such a thing? Well, one part is easy: figuring out to cast automation behaviors with the device in terms of Webdriver commands. [A few years ago](https://www.youtube.com/watch?v=e61OhZzbsEI), I imagined what an Appium test for an X-Wing might look like:\n\n

\n \"Fire\n

\n\nPretty much any interface, virtual or physical, can be cast in terms of elements and interactions with them. So the _real_ question is, how do you turn Webdriver commands into actual behaviors for a piece of hardware? That is the subject of this series, and in the second part we will answer that question directly. For this first part, we need to first develop the \"app under test\" to prove out the whole idea. In other words, we need to build ourselves a physical device that we will then write an Appium test for later on.\n\nThe device we'll build is pictured at the top, and it is a drum machine! Here are the requirements for this \"app\" (device):\n\n1. It should have 4 buttons as input\n2. It should make 4 sounds as output---one sound corresponding to each button\n3. Sounds should be sent along an audio cable for use with headphones or speakers\n4. As additional feedback, there should also be visual (LED) output whenever a button is pressed\n\nThat's exactly how this little machine works. Here's a GIF of it in action (of course, you'll have to imagine the sound---and believe me that your imagination will be better than the quality of the actual samples):\n\n

\n \"Animated\n

\n\nAnd here's how we build it:\n\n1. Buy an [Adafruit Circuit Playground Express](https://www.adafruit.com/product/3333) (CPX). This is the microcontroller that will be in charge of detecting button presses, playing audio, and even choosing when to turn an LED from red to green.\n2. Buy 4 [large buttons](https://www.adafruit.com/product/1192) (these ones include LEDs but we will not use that functionality), a bunch of [alligator clips](https://www.adafruit.com/product/1008), and a 1Kohm resistor (or potentiometer).\n3. Hook up each button to a ground pad on the CPX, and the button lead to one of the input pads on the CPX (I used A1, A2, A5, and A6).\n4. Take a wire from AO (audio out) to the resistor, and from the resistor to your headphone jack (make sure it touches either side of the outermost separator, in other words, touching both of the outermost metal areas). Take another wire from a ground pad on the CPX to the innermost metal segment on your headphone jack.\n\nNow that it's all built, we need to give it some brains! Ensure that your CPX has been [set up to work with CircuitPython](https://learn.adafruit.com/adafruit-circuit-playground-express/circuitpython-quickstart). Then, check out the [source files](https://github.com/jlipps/appiumconf2019/tree/master/cpx) which need to be copied to the CPX (using `rsync` or simply drag-and-drop). Once the code is copied, the CPX will reboot, and you should have a functional drum machine!\n\nThis drum machine is not so good, and has a bunch of limitations. It can only play one sound at a time, so you can't hit the bass and the cymbal simultaneously, for example. The sound quality is ridiculously bad. But it works!\n\nThe only thing left to understand in this part of the series is what the \"app code\" looks like. You can see that there are first of all 4 WAV files, corresponding to the sounds produced by the button. The code itself lives in [`code.py`](https://github.com/jlipps/appiumconf2019/blob/master/cpx/code.py):\n\n```py\nfrom adafruit_circuitplayground.express import cpx\nfrom digitalio import DigitalInOut, Direction, Pull\nimport audioio\nimport board\n\n\nINPUTS = [\n {'board': board.A1, 'pixel': 6, 'drum': 'kick-8'},\n {'board': board.A2, 'pixel': 8, 'drum': 'snare-8'},\n {'board': board.A5, 'pixel': 1, 'drum': 'hihat-8'},\n {'board': board.A6, 'pixel': 3, 'drum': 'hitom-8'},\n]\n\n\ndef play_file(filename):\n wave_file = open(filename, \"rb\")\n with audioio.WaveFile(wave_file) as wave:\n with audioio.AudioOut(board.A0) as audio:\n print(\"playing %s\" % filename)\n audio.play(wave)\n while audio.playing:\n pass\n\n\ndef init():\n for input_spec in INPUTS:\n pin = DigitalInOut(input_spec['board'])\n pin.direction = Direction.INPUT\n pin.pull = Pull.UP\n input_spec['pin'] = pin\n\n cpx.pixels.brightness = 0.1\n\n\ninit()\n\n\nwhile True:\n for input_spec in INPUTS:\n if input_spec['pin'].value:\n cpx.pixels[input_spec['pixel']] = (255, 0, 0)\n else:\n cpx.pixels[input_spec['pixel']] = (0, 255, 0)\n play_file(\"%s.wav\" % input_spec['drum'])\n```\n\nThe important things to note here are:\n\n* We are using an Adafruit-specific library to gain access to a bunch of helpful functionality. This is how we easily turn WAV files into electrical outputs, or control the LEDs. Without this built-in library, our project would be _much_ harder.\n* We keep the code clean and general by defining a mapping of input pads, sounds, and LED identifiers.\n* The code runs in an infinite loop. On each iteration of the loop, we simply check whether a button is pressed (i.e., `input_spec['pin'].value` is False, meaning the button has broken the circuit by being pressed down). If so, we set the corresponding LED to green, and play the corresponding WAV file.\n\nThat's it! In less than 50 lines of code, and without breaking the bank (this setup cost around $50USD, but of course all the parts are reusable for other projects), we've built our very own light-animated drum machine! You can see that I chose to \"mount\" my drum machine in a used yogurt container. You might choose something more classy---that's up to you. Be sure to check out Part 2 of this series to see how we tackle the challenge of writing Appium tests for this custom hardware device.\n","metadata":{"lang":"Python","platform":"Circuit Playground Express","subPlatform":"All Devices","title":"Automating Custom IoT Devices With Appium, Part 1","shortDesc":"Appium is useful for more than just automation of mobile apps. In fact, Appium can be used to automate platforms that have no visual UI at all, such as custom IoT devices or electronics project. In this series, inspired by the AppiumConf 2019 keynote, we look at how to use Appium to test a completely custom piece of electronics hardware.","liveAt":"2019-06-19 10:00","newsletterTopNote":"In recognition of the year anniversary of AppiumConf 2019 (and the fact that we're not having a conference this year due to COVID-19), I'm reposting this 2-part series on how to use Appium to test IoT devices! I think it's a super interesting topic, so please enjoy something a little ... different today.","canonicalRef":"https://www.headspin.io/blog/automating-custom-iot-devices-with-appium-part-1"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/74-automating-custom-iot-devices-with-appium-part-1","slug":"74-automating-custom-iot-devices-with-appium-part-1","path":"/editions/74-automating-custom-iot-devices-with-appium-part-1","news":{"rawMd":"Speaking of music and Appium, do you remember that time I had Appium [automate software instruments](https://www.youtube.com/watch?v=zsbNVkayYRQ) while I played guitar and sang about robots ruining our lives? Or that other time I had Appium [automate even more instruments](https://www.youtube.com/watch?v=29c22KqtG3s) while I played ukulele and same about _another_ techno-dystopian scenario? It's like a theme, or something! Well, Appium is great, but to be honest, it's not the best musician. One of the other things I occasionally do in life is write and record music, so of course I had to put together \"real\" versions of these songs, since I liked them so much (sorry Appium). My band goes by the name of Splendour Hyaline and we just released a record that took us over 11 years to produce! Both of the Appium songs are on there, with a bunch of other indie pop/rock tunes. My musical alter ego would appreciate you checking them out! Go ahead and have a listen at [Splendour Hyaline's Bandcamp page](https://splendourhyaline.bandcamp.com/album/present-future) (there's a free sampler version of the album available there as well).\n\n_OK, back to hacking on Appium. Next music promotion in T minus 11 years!_\n","num":74,"mdPath":"/vercel/path0/content/news/0074.md"},"tags":["Python","Circuit Playground Express","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0073.md","num":73,"rawMd":"\n\n

\n \"App\n

\n\nIt's not very common, but it can happen that an Android hybrid app has not just one but two or more webviews (for example, when advertisements are in one webview and app logic is in another). If you've ever found yourself in this situation, you may have been frustrated when using Appium's context API, specifically with code like this:\n\n```java\nSet handles = driver.getContextHandles();\n```\n\nIf you have multiple webviews, you might have expected the Set to contain three items: `NATIVE_APP`, and two webview-looking strings. But instead you probably found only _one_ webview context. If you were lucky, it was the one you wanted, but there was an equal chance you got the unimportant advertising webview (or whatever). So, how can we be sure to get inside the particular webview we want, even when there are multiple?\n\nThe answer lies in how Chromedriver works. Appium uses Chromedriver for Chrome automation, and Chromedriver handles multiple webviews in a single Android app by treating them as separate _windows_. What this means is that we have access to them using the built-in Webdriver window commands, like so:\n\n```java\nSet windowHandles = driver.getWindowHandles();\n```\n\nThen, we can switch into any window we want by picking a certain handle:\n\n```java\ndriver.switchTo().window(handle);\n```\n\nHow do we know from the handle which window to choose? We don't, because it will be a random-looking string like `CDwindow-A26869B5EDAEAA9D83947BB274F1D0C7`, and it might change from test to test. So our best bet is to switch into each one and perform some validation on the window, to prove that we are in the correct one (for example, looking at the URL or title as a way of ensuring we're in the right webview, and short-circuiting our search when we find it).\n\nOf course, to do any of this we actually have to be in the webview _context_ first, otherwise the `getWindowHandles` command won't do anything. So to conclude, here's a full example that utilizes a new feature of The App, namely a view with two webviews. The idea behind this test is simply to make an assertion based on the contents of the page displayed in each webview, proving that we can indeed enter and automate both of them. As with [Edition 17](https://appiumpro.com/editions/17), we use a helper function `getWebContext` to get ourselves into the initial web context so that Chromedriver is operative. Without further ado, here's the full sample:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.AndroidDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.ArrayList;\nimport javax.annotation.Nullable;\nimport org.hamcrest.Matchers;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition073_Multiple_Webviews {\n\n private String APP_ANDROID = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.10.0/TheApp-v1.10.0.apk\";\n\n private AndroidDriver driver;\n private static By hybridScreen = MobileBy.AccessibilityId(\"Dual Webview Demo\");\n private static By webview = By.className(\"android.webkit.WebView\");\n\n @Before\n public void setUp() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", APP_ANDROID);\n\n driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testDualWebviews_Android() {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n // get to dual webview screen and make sure it's loaded\n wait.until(ExpectedConditions.presenceOfElementLocated(hybridScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(webview));\n\n // navigate to the webview context\n String webContext = getWebContext(driver);\n driver.context(webContext);\n\n // go into each available window and get the text from the html page\n ArrayList webviewTexts = new ArrayList<>();\n for (String handle : driver.getWindowHandles()) {\n System.out.println(handle);\n driver.switchTo().window(handle);\n webviewTexts.add(driver.findElement(By.tagName(\"body\")).getText());\n }\n\n // assert that we got the correct text from each android webview\n Assert.assertThat(webviewTexts,\n Matchers.containsInAnyOrder(\"This is webview '1'\", \"This is webview '2'\"));\n }\n\n @Nullable\n private String getWebContext(AppiumDriver driver) {\n ArrayList contexts = new ArrayList(driver.getContextHandles());\n for (String context : contexts) {\n if (!context.equals(\"NATIVE_APP\")) {\n return context;\n }\n }\n return null;\n }\n}\n```\n\nThat's it! With judicious use of both context and window handles commands, automating multiple webviews is totally doable. Don't forget to check out the [full code](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition073_Multiple_Webviews.java) on GitHub.\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Working with Multile Webviews in Android Hybrid Apps","shortDesc":"Some Android apps are built with multiple webviews, and they're a bit tricky to work with because they don't show up in the context list as you'd expect. Thankfully there's an easy workaround which enables automation of any number of webviews.","liveAt":"2019-06-12 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-multile-webviews-in-android-hybrid-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/73-working-with-multile-webviews-in-android-hybrid-apps","slug":"73-working-with-multile-webviews-in-android-hybrid-apps","path":"/editions/73-working-with-multile-webviews-in-android-hybrid-apps","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0071.md","num":71,"rawMd":"\n\nReader `kuznietsov.com` suggested that we write an article introducing the AppiumServiceBuilder functionality built into the Appium Java client.\n\nAll of our example code assumes that an Appium server is already running, and this is how many test suites start out too. It's so much more convenient if an Appium server is started automatically when the tests begin. The Java client has a convenient couple of classes for starting and stopping an Appium server. Other languages may have separate packages for doing this, or you can execute the `appium` command as a separate process.\n\nNode.js has the advantage of being the language Appium is written in, so an Appium server can be started just by `require`ing it:\n```javascript\nlet Appium = require('appium')\nlet server = await Appium.main()\n// server now running\nawait server.close()\n// server stopped\n```\n\nBack in Java land, we start by creating an `AppiumServiceBuilder` which we then use to create an `AppiumDriverLocalService`:\n```java\nAppiumServiceBuilder serviceBuilder = new AppiumServiceBuilder();\n// some additional setup may be needed here, keep reading for more details\nserver = AppiumDriverLocalService.buildService(serviceBuilder);\n```\n\nNow that we have an `AppiumDriverLocalService` named `server`, we can start and stop it easily. When starting, an Appium server will run, and you will be able to connect to it. The Appium server logs, by default, are printed in the output of the test.\n\n```java\nserver.start();\n// you can create connect clients and automate tests now\n```\n\nDon't forget to stop the server when you are done:\n\n```java\nserver.stop()\n```\n\nIn order to run an Appium server, the Java code in the AppiumServiceBuilder needs to know the location of the Node.js executable on your computer, and also the location of the Appium package itself. It has ways to guess the location, but those did not work on my machine, since I installed Node.js using [nvm](https://github.com/nvm-sh/nvm). If you only have Appium Desktop installed, you will also have to install the Appium node package, so it can be run from the commandline.\n\n```java\nAppiumServiceBuilder serviceBuilder = new AppiumServiceBuilder();\nserviceBuilder.usingDriverExecutable(new File(\"/path/to/node/executable\"));\nserviceBuilder.withAppiumJS(new File(\"/path/to/appium\"));\n```\n\nIf you share your test code with a team and/or run in different CI environments, you can set this information via environment variables rather than hardcoding file paths in your test code. Store the paths in environment variables named `NODE_PATH` and `APPIUM_PATH`, and the AppiumServiceBuilder will pick them up automatically.\n\nAnother issue I ran into is that the XCUITest driver requires the Carthage package manager to be in the system PATH. If you followed the installation instructions for XCUITest driver, Carthage is installed using Homebrew and it is automatically added to the PATH, so this isn't normally an issue. Unfortunately, the way that AppiumDriverLocalService runs Appium, it does not re-use your defauly PATH and instead reverts to the system default. For my case, I corrected this by specifying a new PATH environment variable in the AppiumServiceBuilder:\n\n```java\nHashMap environment = new HashMap();\nenvironment.put(\"PATH\", \"/usr/local/bin:\" + System.getenv(\"PATH\"));\nserviceBuilder.withEnvironment(environment);\n```\n\nYou can add other environment variables for Appium using this same method. You can also specify other options, such as custom desired capabilities that Appium should default to, and custom ways of exporting Appium logs.\n\nFor my tests, I decided to use `serviceBuilder.usingAnyFreePort();`, so that when my tests run, the Appium server will use any available port, rather than insisting on using port 4723. This way, if I have a forgotten Appium server running in another window, it won't interfere with my tests.\n\nIf Appium starts on a random port, our code to start a session needs to know what the port is so the client can connect to it. To solve this, I connect the client this way:\n\n```java\nserviceBuilder.usingAnyFreePort();\nserver.start();\ndriver = new IOSDriver(server.getUrl(), caps);\n```\n\nThe AppiumDriverLocalService object has a `getUrl()` method which will return the URL and port of the Appium server it started.\n\nThere are more methods you can use to customize the Appium server programatically. Check out the documentation here and try experimenting: https://appium.github.io/java-client/io/appium/java_client/service/local/AppiumServiceBuilder.html\n\nHere's the full example, putting all this together for a sample test. As usual the [full example code is also available on github](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition071_AppiumServiceBuilder.java).\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileElement;\nimport io.appium.java_client.ios.IOSDriver;\nimport io.appium.java_client.service.local.AppiumDriverLocalService;\nimport io.appium.java_client.service.local.AppiumServiceBuilder;\nimport org.junit.*;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\nimport java.io.File;\nimport java.util.HashMap;\n\nimport static junit.framework.TestCase.assertTrue;\n\npublic class Edition071_AppiumServiceBuilder {\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip\";\n\n private AppiumDriver driver;\n private static AppiumDriverLocalService server;\n\n @BeforeClass\n public static void startAppiumServer() {\n AppiumServiceBuilder serviceBuilder = new AppiumServiceBuilder();\n // Use any port, in case the default 4723 is already taken (maybe by another Appium server)\n serviceBuilder.usingAnyFreePort();\n // Tell serviceBuilder where node is installed. Or set this path in an environment variable named NODE_PATH\n serviceBuilder.usingDriverExecutable(new File(\"/Users/jonahss/.nvm/versions/node/v12.1.0/bin/node\"));\n // Tell serviceBuilder where Appium is installed. Or set this path in an environment variable named APPIUM_PATH\n serviceBuilder.withAppiumJS(new File(\"/Users/jonahss/.nvm/versions/node/v12.1.0/bin/appium\"));\n // The XCUITest driver requires that a path to the Carthage binary is in the PATH variable. I have this set for my shell, but the Java process does not see it. It can be inserted here.\n HashMap environment = new HashMap();\n environment.put(\"PATH\", \"/usr/local/bin:\" + System.getenv(\"PATH\"));\n serviceBuilder.withEnvironment(environment);\n\n server = AppiumDriverLocalService.buildService(serviceBuilder);\n server.start();\n }\n\n @Before\n public void startSession() {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.2\");\n caps.setCapability(\"deviceName\", \"iPhone Xs\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n\n caps.setCapability(\"app\", APP);\n\n driver = new IOSDriver(server.getUrl(), caps);\n }\n\n @After\n public void endSession() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @AfterClass\n public static void stopAppiumServer() {\n server.stop();\n }\n\n @Test\n public void test() {\n // test code goes here\n assertTrue(true);\n }\n}\n```\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Starting an Appium Server Programmatically Using AppiumServiceBuilder","shortDesc":"Learn how to use the AppiumServiceBuilder class, which is built into the Appium java client to provide an easy way of starting and stopping an Appium server from within test code.","liveAt":"2019-05-29 10:00","canonicalRef":"https://www.headspin.io/blog/starting-an-appium-server-programmatically-using-appiumservicebuilder"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/71-starting-an-appium-server-programmatically-using-appiumservicebuilder","slug":"71-starting-an-appium-server-programmatically-using-appiumservicebuilder","path":"/editions/71-starting-an-appium-server-programmatically-using-appiumservicebuilder","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0070.md","num":70,"rawMd":"\n\n

\n \"Buy\n

\n\nPreviously, we looked at [how to capture audio playback during testing](https://appiumpro.com/editions/69). Now, it's time to look at how to verify that the audio matches our expectations! The very fact that we have expectations needs to be expressed in audio form. In other words, what we want to do is take the audio we've captured from a particular test run, and assert that it is in some way _similar_ to another audio file that we already have available. We can call this latter audio file the \"baseline\" or \"gold standard\", against which we will be running our tests.\n\n### Context\n\nIn the first part, the audio we care about verifying is a sound snippet from one of [my band's old songs](https://www.amazon.com/Restless-Slumber-Dry-Kindling/dp/B00MU8WJHW). So what we need to do is save a snippet we know to be \"good\", so that future versions of the tests can be compared against this. I've gone ahead and copied such a snippet into the [resources directory](https://github.com/cloudgrey-io/appiumpro/tree/master/java/src/test/resources) of the Appium Pro project. The state of our test as we left it in the previous part was that we had captured audio from an Android emulator, and asserted the audio had been saved, but had not done anything with it. Here's where we starting out from now, with our new test class that inherits from the old test class:\n\n```java\npublic class Edition070_Audio_Verification extends Edition069_Audio_Capture {\n\n private File getReferenceAudio() throws URISyntaxException {\n URL refImgUrl = getClass().getClassLoader().getResource(\"Edition070_Reference.wav\");\n return Paths.get(refImgUrl.toURI()).toFile();\n }\n\n @Test\n @Override\n public void testAudioCapture() throws Exception {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n // navigate to band homepage\n driver.get(\"http://www.splendourhyaline.com\");\n\n // click the amazon store icon for the first album\n WebElement store = wait.until(ExpectedConditions.presenceOfElementLocated(\n By.cssSelector(\"img[src='/img/store-amazon.png']\")\n ));\n driver.executeScript(\"window.scrollBy(0, 100);\");\n store.click();\n\n // start playing a sample of the first track\n WebElement play = wait.until(ExpectedConditions.presenceOfElementLocated(\n By.xpath(\"//div[@data-track-number='2']//div[@data-action='dm-playable']\")\n ));\n\n driver.executeScript(\"window.scrollBy(0, 150);\");\n\n // start the song sample\n play.click();\n\n // start an ffmpeg audio capture of system audio. Replace with a path and device id\n // appropriate for your system (list devices with `ffmpeg -f avfoundation -list_devices true -i \"\"`\n File audioCapture = new File(\"/Users/jlipps/Desktop/capture.wav\");\n captureForDuration(audioCapture, 10000);\n }\n}\n```\n\nWhat we now need to do is assert that our `audioCapture` File in some sense matches our gold standard. But how on earth would we do that?\n\n### Audio file similarity\n\nAs a naive approach, we could assume that similar wav files might be similar on a byte level. We could try to use something like an MD5 hash of the file and compare it with our gold standard. This, however, will not work. Unless the WAV files are exactly the same, the MD5 hash will likely be completely unrelated. We could get slightly more complicated, and actually read the WAV file as a stream of bytes, and compare each byte of our captured WAV with the baseline WAV. This approach, unfortunately, is also doomed to fail! Tiny differences in sound would lead to huge differences on a byte level. Also, if the timing of the two WAV files differs by anything more than the sample rate (which is many thousands of times per second), every single byte will be different and our comparison will be utter garbage.\n\nWhat we will do instead is take advantage of work that has been done in the world of audio _fingerprinting_. Fingerprinting is what lies behind services like Shazam or last.fm that can detect what song is being played even though it might be recorded through your phone's microphone. Fingerprinting is a complex algorithm that takes into account various acoustic properties of WAV file segments, and produces what is essentially a hash of a piece of audio. The important thing is that similar audio files will produce more similar hashes, so they can actually be fruitfully compared with one another.\n\n### Chromaprint\n\nThe fingerprinting library we will use is called [Chromaprint](https://acoustid.org/chromaprint), and you will need to download the appropriate version for your system. Just like with ffmpeg, we will run the Chromaprint binary as a Java subprocess. The way we'd run it outside of Java, on the command line, would be like this:\n\n```\n# in the chromaprint directory\n./fpcalc -raw /path/to/audio.wav\n```\n\nThis will produce output that corresponds to the fingerprint of the audio file. Using the `-raw` flag means we get the raw numeric output rather than the base64-encoded output (which is nice and small, but makes the comparisons between fingerprints less strong). Running from the terminal, the output will look something like:\n\n```\nDURATION=9\nFINGERPRINT=1663902633,1696875689,1709563129,1688656920,1688648712,1688644617,1688708873,1692705291,1684312587,1814528570,1832305258,1863766762,3993379306,3993380074,3997640938,3998550234,4275505354,4271783130,4137558234,1988825290,1451753930,1586000842,1586029546,3599411178,3603677163,3595079099,3590884507,1444650121,1448785096,1985647688,1985647640,1985581096,1983422504,1983438904,2000576584,1949176009,1949172170,1950220986,1954481834,1958288106,1958292202,1958247034,1957227082,1957211979,2011735113,1982377992,2116780040,2117763128,1041928232,1041948712,907960360,907878456,1728520216,1812338040,1812336376,1818103480\n```\n\nBut we want to run this from Java, so we need a handy class that encapsulates all this fingerprinting business, including running Chromaprint's `fpcalc` binary. It will also be responsible for parsing the response and storing it in a way that makes comparison easy:\n\n```java\nclass AudioFingerprint {\n\n private static String FPCALC = \"/Users/jlipps/Desktop/chromaprint/fpcalc\";\n\n private String fingerprint;\n\n AudioFingerprint(String fingerprint) {\n this.fingerprint = fingerprint;\n }\n\n public String getFingerprint() { return fingerprint; }\n\n public double compare(AudioFingerprint other) {\n return FuzzySearch.partialRatio(this.getFingerprint(), other.getFingerprint());\n }\n\n public static AudioFingerprint calcFP(File wavFile) throws Exception {\n String output = new ProcessExecutor()\n .command(FPCALC, \"-raw\", wavFile.getAbsolutePath())\n .readOutput(true).execute()\n .outputUTF8();\n\n Pattern fpPattern = Pattern.compile(\"^FINGERPRINT=(.+)$\", Pattern.MULTILINE);\n Matcher fpMatcher = fpPattern.matcher(output);\n\n String fingerprint = null;\n\n if (fpMatcher.find()) {\n fingerprint = fpMatcher.group(1);\n }\n\n if (fingerprint == null) {\n throw new Exception(\"Could not get fingerprint via Chromaprint fpcalc\");\n }\n\n return new AudioFingerprint(fingerprint);\n }\n}\n```\n\nBasically, what's going on here is that we are setting a path to the `fpcalc` binary, and then using the `ProcessExecutor` Java library (from the good folks at [ZeroTurnaround](https://github.com/zeroturnaround/zt-exec) to make executing `fpcalc` very easy. We then use regular expression matching on the output to extract a fingerprint from an audio file. Most of the code here is simply Java class boilerplate and regular expression logic!\n\n### Comparing fingerprints\n\nThe most important bit is the `compare` method, where we are making use of something called the [Levenshtein distance](https://en.wikipedia.org/wiki/Levenshtein_distance) between strings to figure out how similar to audio fingerprints really are. To this end I'm using a library called [JavaWuzzy](https://github.com/xdrop/fuzzywuzzy) (a port of the useful Python library [FuzzyWuzzy](https://github.com/seatgeek/fuzzywuzzy)), which contains the important algorithms so I don't need to worry about implementing them. The response of my call to the `partialRatio` method is a number between 0 and 100, where 100 is a perfect match and 0 signifies no matching segments at all.\n\nAll we need to do then, is hook this class up into our test so that we can fingerprint both our newly-captured audio as well as the baseline audio, and then run the comparison. In my experiments, I was able to achieve a value of about 75 for a correct comparison, whereas other song snippets came in at an appropriately lower value, say 45. Of course you'll want to determine through experimentation what your similarity threshold should be, based on the particular audio domain, clip length, etc...\n\nHooking in the new code is relatively easy (starting from the point in the test method where we have the `audioCapture` file populated with the new audio:\n\n```java\n// now we calculate the fingerprint of the freshly-captured audio...\nAudioFingerprint fp1 = AudioFingerprint.calcFP(audioCapture);\n\n// as well as the fingerprint of our baseline audio...\nAudioFingerprint fp2 = AudioFingerprint.calcFP(getReferenceAudio());\n\n// and compare the two\ndouble comparison = fp1.compare(fp2);\n\n// finally, we assert that the comparison is sufficiently strong\nAssert.assertThat(comparison, Matchers.greaterThanOrEqualTo(70.0));\n```\n\nHere, I've added a helper method called `getReferenceAudio()` to get me the baseline audio `File` object from the resources directory. And notice the assertion in the final line, which turns this bit of automation into a bona fide test of audio similarity!\n\nSo, when all is said and done, it __is__ possible to test audio with Appium and Java (and since we are using ffmpeg and Chromaprint as subprocesses, the same technique can be used in any other programming language as well). This is relatively unexplored territory, though, so I would expect there to be a certain amount of potential flakiness for this kind of testing. That being said, the Chromaprint fingerprinting algorithm is used commercially and appears to be quite good, so at the end of the day the quality of the test will depend on the quality, length, and genre of your audio. Please do let me know if you put this into practice as I'd love to hear any case studies of this technique. And don't forget to check out the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition070_Audio_Verification.java) on GitHub, to see everything in context. Happy testing, and happy listening! Oh, and in case you really wanted to know: yes, my band will be coming out with a new studio album very soon!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Capturing Audio Output During Testing: Part 2","shortDesc":"In the second part of this audio testing series, we look at the all-important step of audio verification, which boils down to determining how similar two different audio files are. This is no easy task, but thanks to the existence of audio fingerprinting libraries, it's one we can handle.","sponsor":"headspin","sponsorLink":"https://www.headspin.io/audio-visual-platform","sponsorBlurb":"HeadSpin's custom bluetooth board and analysis API allows testing voice assistants, validating streaming media, and working with voice calls on real devices.","liveAt":"2019-05-22 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-audio-output-during-testing-part-2"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/70-capturing-audio-output-during-testing-part-2","slug":"70-capturing-audio-output-during-testing-part-2","path":"/editions/70-capturing-audio-output-during-testing-part-2","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0069.md","num":69,"rawMd":"\n\n

\n \"Buy\n

\n\nOne of the most common observations you can make while watching people using their mobile apps in the real world is that they often have headphones plugged into them (or not, depending on their level of [courage](https://www.engadget.com/2018/09/12/apple-iphone-headphone-jack-se-6s/)). We could talk more about extreme headphone usage as a social phenomenon, but this is Appium Pro! As testers, we're thinking to ourselves, \"Oh no, have we tested the audio playback portion of our app?\"\n\nAudio playback testing is a hard concept to tackle, so we're going to explore it over 2 editions. In this first edition, we'll examine audio _capture_, or how to make sure we can get audio files generated from real user scenarios in our app. In the second part, we'll look at audio _verification_, which is the real important piece. But of course, we can't verify audio playback if we have no way to capture the audio! So let's figure this out.\n\nWe have several different options when it comes to collecting audio, and the first is to choose whether we're capturing from a real device or a virtual device.\n\n### Capturing audio from a real device\n\nTo capture audio from a real mobile device, you'll need a device that has a headphone jack or adapter. Plug one end of a 3.5mm male-to-male headphone cable into the headphone jack, and the other into your digital audio input device. Many computers these days don't support line in directly, so you may need an external digital audio converter. Either way, remember which audio input device the headphone is now connected to, and simply skip down to the ffmpeg setup instructions below.\n\n### Capturing audio from a virtual device\n\nVirtual devices play audio using the system audio outputs, so for virtual devices we need a way to treat a system audio output as a kind of audio _input_. I'm going to explain how to do this on a Mac; for other platforms, you'll need to research how to achieve the same result.\n\n#### Soundflower Setup\n\nThe first thing we need to do is get [Soundflower](https://github.com/mattingalls/Soundflower) up and running. This is a kernel extension which adds a special audio output device and a special audio input device called `Soundflower (2ch)`. So go ahead and follow the Soundflower installation instructions until this device is visible in your system (restart probably required). To check whether the device is visible, go to the \"Sound\" preference pane in System Preferences, and you should see the devices under the \"Output\" and \"Input\" tabs.\n\nNow, experiment with playing an audio file. While it's playing, change the output device to the Soundflower output. You should hear the audio cut out. That's because the audio is now being redirected to the Soundflower device, which is not connected to your speakers or headphones. It's a virtual audio device, which we'll use to record sound coming from virtual mobile devices!\n\n#### Android Emulator Setup\n\nAndroid Emulators are a bit weird when it comes to audio. They set their system audio output channel when they boot up for the first time, and at that point _only_. So they do not respect changes you make in the Sound preferences of your Mac. What this means is that we want to make sure our audio settings are set up for internal audio capture via Soundflower _before_ we launch our emulator. And not just before we launch it, but before it _boots_. This means that if you have an already-warm emulator, you'll need to completely shut it down, change sound preferences, and then perform a cold boot on the emulator.\n\nIn addition to the Soundflower selection described above, you'll also want to go to the \"Sound Effects\" tab of the Sound preferences, and make sure that the output is set to mirror the system output. That's because some emulators send all their sound through the Sound Effects channel, for some inexplicable reason.\n\n#### iOS Simulator Setup\n\nGood news---no additional setup is required for iOS!\n\n### ffmpeg Setup\n\nTo actually capture audio, we're going to use a system library called [ffmpeg](https://ffmpeg.org/). ffmpeg does _lots_ of stuff, but we only care about its ability to read from an audio device and output an audio file. Our output file is going to be a standard PCM WAV file, which will be great when it comes to doing the verification step later on. The easiest way to get ffmpeg on a Mac is via the Homebrew command `brew install ffmpeg`, but go ahead and read the ffmpeg docs to make sure it's installed appropriately on your system. To test things, out, run the command:\n\n```\nffmpeg -f avfoundation -list_devices true -i \"\"\n```\n\nThis command looks at the Apple audio devices that are available and lists them out, with corresponding IDs. This is the last part of the output I get on my machine:\n\n```\n[AVFoundation input device @ 0x7f9427501240] AVFoundation video devices:\n[AVFoundation input device @ 0x7f9427501240] [0] HD Pro Webcam C920\n[AVFoundation input device @ 0x7f9427501240] [1] FaceTime HD Camera\n[AVFoundation input device @ 0x7f9427501240] [2] Capture screen 0\n[AVFoundation input device @ 0x7f9427501240] AVFoundation audio devices:\n[AVFoundation input device @ 0x7f9427501240] [0] Built-in Microphone\n[AVFoundation input device @ 0x7f9427501240] [1] HD Pro Webcam C920\n[AVFoundation input device @ 0x7f9427501240] [2] USB audio CODEC\n[AVFoundation input device @ 0x7f9427501240] [3] Soundflower (64ch)\n[AVFoundation input device @ 0x7f9427501240] [4] Soundflower (2ch)\n```\n\nIn our case, we don't care about video devices, only audio devices. And the one I care about in particular is `[4] Soundflower (2ch)` (since I'm recording virtual device audio---make adjustments according to your situation if you're recording from an actual physical input). So I'll need to remember that, from ffmpeg's perspective, the Soundflower audio device is number `4`. Now, we _could_ use ffmpeg from the command line to capture audio, but that doesn't do us much good in the context of an automated test. What we really want to do is call ffmpeg from inside our test code! So now let's discuss how to integrate all of these things within an Appium test to achieve audio capture.\n\n### Capturing Audio in a Test\n\nWhat we're going ultimately going to test in this mini series is that the Amazon Music sample audio clip for a song from my band's old album actually sounds correct. In this edition, we're merely going to work on capturing a portion of the audio sample. Here are our test steps in bullet form:\n\n* Navigate to [http://splendourhyaline.com](http://splendourhyaline.com) (my hastily-coded band page)\n* Click the first link to buy the album on Amazon\n* Find the second song in the song list, and click the \"play\" button to start playing a sample of it\n* At this point, turn on ffmpeg to record audio from the Soundflower device, which should be streaming the audio from the Android emulator\n* Wait enough time to capture a good amount of audio\n* Stop the ffmpeg recording\n* Verify that the ffmpeg recording in fact generated a WAV file\n\nAnd then we'll have a `TODO` for next time which will be all about comparing a captured WAV file with a gold standard, to make sure the sound was what we expected. In this case, we'll be capturing music, but we could be capturing text-to-speech, voice assistant output, or whatever's appropriate for your app.\n\nLet's first have a look at the standard Appium web automation code:\n\n```java\nWebDriverWait wait = new WebDriverWait(driver, 10);\n\n// navigate to band homepage\ndriver.get(\"http://www.splendourhyaline.com\");\n\n// click the amazon store icon for the first album\nWebElement store = wait.until(ExpectedConditions.presenceOfElementLocated(\n By.cssSelector(\"img[src='/img/store-amazon.png']\")\n));\ndriver.executeScript(\"window.scrollBy(0, 100);\");\nstore.click();\n\n// start playing a sample of the first track\nWebElement play = wait.until(ExpectedConditions.presenceOfElementLocated(\n By.xpath(\"//div[@data-track-number='2']//div[@data-action='dm-playable']\")\n));\n\ndriver.executeScript(\"window.scrollBy(0, 150);\");\n\n// start the song sample\nplay.click();\n```\n\nBasically all this code just gets us to the point where we should actually be hearing music playing. What we want now is to start capturing audio to a file, and then assert that that file actually exists when done. Here's how we're going to do this on a relatively abstract level:\n\n```java\nFile audioCapture = new File(\"/Users/jlipps/Desktop/capture.wav\");\ncaptureForDuration(audioCapture, 10000);\n\nassert(audioCapture.exists());\n```\n\nIn other words, I define a place on my system to capture the audio (of course you'll want an appropriate path for your system), then I call a helper method `captureForDuration`, into which I pass my desired audio file path, and the amount of time I want to capture audio for (in this case 10 seconds). After this, I simply make an assertion that the captured file exists. (In the next edition, we'll update this to be a verification of the content of the audio file).\n\nHow do we go about implementing `captureForDuration`? Well, first of all we know that we'll need to run ffmpeg as a sub-process. That means dealing with Threads, Runnables, and ProcessBuilder! (At least in Java---every language has its own specific way of dealing with subprocesses). What I've done, therefore, is to create an `FFmpeg` class to encapsulate all these responsibilities:\n\n```java\nclass FFmpeg implements Runnable {\n private Process proc;\n private File captureFile;\n private int deviceId;\n\n FFmpeg(File captureFile, int deviceId) {\n this.proc = null;\n this.captureFile = captureFile;\n this.deviceId = deviceId;\n }\n\n public void run() {\n ArrayList cmd = new ArrayList<>();\n cmd.add(\"ffmpeg\"); // binary should be on path\n cmd.add(\"-y\"); // always overwrite files\n cmd.add(\"-f\"); // format\n cmd.add(\"avfoundation\"); // apple's system audio---something else for windows\n cmd.add(\"-i\"); // input\n cmd.add(\":\" + deviceId); // device id returned by ffmpeg list\n cmd.add(captureFile.getAbsolutePath());\n\n ProcessBuilder pb = new ProcessBuilder(cmd);\n pb.redirectErrorStream(true);\n pb.redirectOutput(Redirect.PIPE);\n StringJoiner out = new StringJoiner(\"\\n\");\n try {\n proc = pb.start();\n try (BufferedReader reader = new BufferedReader(\n new InputStreamReader(proc.getInputStream()))) {\n\n reader.lines().forEach(out::add);\n }\n proc.waitFor();\n } catch (IOException | InterruptedException ign) {}\n System.out.println(\"FFMpeg output was: \" + out.toString());\n }\n\n public void stopCollection() throws IOException {\n BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(proc.getOutputStream()));\n writer.write(\"q\");\n writer.flush();\n writer.close();\n }\n}\n```\n\nExplaining how this works in detail is beyond the scope of our Appium usage. But basically what we're doing is calling the `ffmpeg` binary (which we assume exists on the system and is available on the path) with the appropriate arguments to start an audio capture. What it would look like if we were running it directly on the command line (assuming a value of `4` for the deviceId---you'll need to use whatever you saw in the previous command output):\n\n```\nffmpeg -y -f avfoundation -i \":4\" /Users/jlipps/Desktop/capture.wav\n```\n\nSo that is the command that we start using `pb.start`. This command would normally run forever (or until we run out of disk space), so we need some way to tell ffmpeg to stop recording. That's the purpose of the `stopCollection` method, which we'll call from outside the class. This method simply sends the letter \"q\" to the process, which (because of the way ffmpeg is designed) stops recording and ends the process. We make sure to wait for the process to end cleanly (using `proc.waitFor()`), and we show the output of the ffmpeg command in the console so if there were any errors we can have a look at them.\n\nWith this class in hand, we're ready to implement `captureForDuration`:\n\n```java\nprivate void captureForDuration(File audioCapture, int durationMs) throws Exception {\n FFmpeg capture = new FFmpeg(audioCapture, 4);\n Thread t = new Thread(capture);\n t.start();\n\n // wait for sufficient amount of song to play\n Thread.sleep(durationMs);\n\n // tell ffmpeg to stop sampling\n capture.stopCollection();\n\n // wait for ffmpeg thread to end on its own\n t.join();\n}\n```\n\nYou can see here that we create an instance of our `FFmpeg` class, and then start the subprocess in a new `Thread`. This allows the ffmpeg process to collect audio data while we wait for a predetermined amount of time. After that time has elapsed, we call the `stopCollection` method to tell ffmpeg to stop collecting data, and then we wait for the subprocess to end before moving on (that's the purpose of `Thread.join`).\n\nObviously, we could make our Thread logic and FFmpeg subprocess handling logic much more robust and safe. But for now, it serves the purpose of showing how we can capture audio in our test for an arbitrary amount of time! Stay tuned for the next edition in which we discuss how to use this captured audio to make verifications about it, which is kind of the whole point! In the meantime, feel free to check out the [full code example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition069_Audio_Capture.java) on GitHub to see all of these pieces put together in a working context. Oh---and don't forget to change your system audio output back to whatever it was before you changed it to Soundflower! Otherwise you won't be able to capture the audio of your favorite music with your own ears!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Capturing Audio Output During Testing: Part 1","shortDesc":"It's not particularly easy, but it is certainly possible to capture and verify audio output for your apps. In the first of this two-part series, we explore the techniques available for capturing audio playback and making it available for later verification.","sponsor":"headspin","sponsorLink":"https://www.headspin.io/audio-visual-platform","sponsorBlurb":"HeadSpin's custom bluetooth board and analysis API allows testing voice assistants, validating streaming media, and working with voice calls on real devices.","liveAt":"2019-05-15 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-audio-output-during-testing-part-1"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/69-capturing-audio-output-during-testing-part-1","slug":"69-capturing-audio-output-during-testing-part-1","path":"/editions/69-capturing-audio-output-during-testing-part-1","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0068.md","num":68,"rawMd":"\n\nSometimes there's an automation feature of apparently little utility, but it's fun all the same. For some reason, Apple decided to allow automation of the physical buttons on an iPhone. Well, some of them: the home button, and the volume up and down buttons. I'm still trying to think of actually useful cases for automating these buttons, but in the meantime, let's look at how to use them.\n\nThey're accessible as one of Appium's `mobile:` methods, like many other non-standard extensions to the WebDriver protocol. Here's an example of how to press the home button:\n\n```java\ndriver.executeScript(\"mobile: pressButton\", ImmutableMap.of(\"name\", \"home\"));\n```\n\nIn other words, there's a single parameter `name`, and it takes one of three values:\n\n* `home`\n* `volumeup`\n* `volumedown`\n\nThe only catch is that the `volumeup` and `volumedown` buttons can only be automated on a _real_ device, not a simulator (for some bizarre reason, since simulators _do_ have virtual volume buttons that can be clicked).\n\nPressing the home button is a good way to get out of your app and back to the home screen. Pressing it twice will navigate back to the _first_ home screen. And pressing it twice in quick succession _should_ open up the app switcher. Unfortunately, even with all my attempts at different timing, I couldn't actually get the app switcher to launch. Nor could I get it to launch manually by clicking the simulator button myself, so it's possible that the simulator's home button recognition is itself not reliable.\n\nAnyway, if any of you readers have an idea of an interesting use case for automating these physical buttons, please do let me know! And you can also check out a [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition068_iOS_Buttons.java) on GitHub if you want to see the command in context.\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Automating Physical Buttons on iOS Devices","shortDesc":"Mobile devices are not always pure screen--they have physical buttons as well. Using Appium's mobile executeScript methods, it's possible to automate several of the physical hardware buttons on an iOS device.","liveAt":"2019-05-08 10:00","canonicalRef":"https://www.headspin.io/blog/automating-physical-buttons-on-ios-devices"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/68-automating-physical-buttons-on-ios-devices","slug":"68-automating-physical-buttons-on-ios-devices","path":"/editions/68-automating-physical-buttons-on-ios-devices","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0067.md","num":67,"rawMd":"\n\nA reader recently asked if we could provide an example of how to zoom in and out using touch gestures. I decided to use Google Maps for the demo, because it is installed by default on Android emulators and thought that this would be a straightforward task. Surprisingly, it ended up being much trickier than I expected for the same reason that Google Maps often frustrates me while I'm driving!\n\nGoogle Maps must implement a lot of custom logic for interpreting touch events. I'm sure multiple usability experts charted people's fingers and intentions, recorded video footage of people smearing finger-oil across screens for hours, and came up with a set of functions to describe the pattern for each gesture. The end result is that our sterile touch actions generated by machines don't trigger the UI reactions we'd expect. The same often happens when I take my life in my hands trying to change the view while driving.\n\n

\n \"Pinch\n

\n\n(I've enabled touch gesture debugging on the emulator, so we can see the gestures that Appium simulates.)\n\nThe simplest gesture composed of two touch inputs (fingers), being placed down on the screen, moved toward or away from each other, and then lifted off the screen, does not budge the UI at all.\n\nWhat I ended up doing was making the gesture more complex, in order to better simulate the more organic and imperfect actions of a real person. I found that the most important variable that led to a successful zooming action was for the gesture to be fast, around 25 to 50 milliseconds. I added a short segment of moving _very_ quickly, followed by a 100ms pause, and then continuing the rest of the zoom gesture in 25-50 milliseconds.\n\n

\n \"Pinch\n

\n\nEven this approach was not very satisfying, as it is not always consistent and the zoom out is more powerful than the zoom in. Plus, I couldn't find a simple way to perform a slow and controlled zoom despite it being so easy to do manually.\n\nWe achieved our goal of demonstrating a zoom gesture though, and there is plenty to learn from in the code used for this.\n\nFirst off, I took advantage of what we learned from our [post about Android intents and activities](https://appiumpro.com/editions/56) to launch the Google Maps app directly to a view of a chosen set of geo coordinates:\n\n```java\nDesiredCapabilities caps = new DesiredCapabilities();\n\ncaps.setCapability(\"platformName\", \"Android\");\ncaps.setCapability(\"deviceName\", \"Android Emulator\");\ncaps.setCapability(\"automationName\", \"UiAutomator2\");\ncaps.setCapability(\"appPackage\", \"com.google.android.apps.maps\");\ncaps.setCapability(\"appActivity\", \"com.google.android.maps.MapsActivity\");\ncaps.setCapability(\"intentAction\", \"android.intent.action.VIEW\");\ncaps.setCapability(\"optionalIntentArguments\", \"-d geo:46.457398,-119.407305\");\n\ndriver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n```\n\nNext, I created a method which would build our \"zoom interaction\". We'll go over the methods used here, but for the history of Appium's gesture API design, and more examples you can visit [a previous article on the topic](https://appiumpro.com/editions/29).\n\nThe new methods for building touch actions compliant with the W3C Webdriver specification are all located under the `org.openqa.selenium.interactions` namespace of the Appium and Selenium Java clients. The end goal is to be able to construct a list of `Interaction` objects which we can then pass to `driver.perform()` in order to send the actions to the Appium server to then run on the device.\n\nSo, my method to create a zoom interaction returns a list of interactions.\n\n```java\nprivate Collection zoom(Point locus, int startRadius, int endRadius, int pinchAngle, Duration duration) {\n // convert degree angle into radians. 0/360 is top (12 O'clock).\n double angle = Math.PI / 2 - (2 * Math.PI / 360 * pinchAngle);\n\n // create the gesture for one finger\n Sequence fingerAPath = zoomSinglefinger(\"fingerA\", locus, startRadius, endRadius, angle, duration);\n\n // flip the angle around to the other side of the locus and get the gesture for the second finger\n angle = angle + Math.PI;\n Sequence fingerBPath = zoomSinglefinger(\"fingerB\", locus, startRadius, endRadius, angle, duration);\n\n return Arrays.asList(fingerAPath, fingerBPath);\n}\n```\n\nEach interaction in the list which we will later pass to `driver.perform()` represents the movement of one finger on the device's touchscreen. A pinch zoom requires two fingers, though the only difference between their movement is the direction relative to the center of the pinch, which I named the \"locus\". `startRadius` and `endRadius` refer to the distance from the locus that the fingers move. `duration` will the the length of time this action takes and `pinchAngle` is how twisted from directly up/down the fingers are while pinching (the examples from earlier all had angles of 45 degrees).\n\nNow let's look at the `zoomSingleFinger` method which actually uses the Appium client methods to create the actions for each finger. The action I decided to make, and which experimentally yielded acceptable results, was to first move the finger very quickly a small distance from the `startRadius` towards the `endRadius`. My finger then pauses for a moment, before resuming its path towards the `endRadius` position.\n\n```java\nprivate Sequence zoomSinglefinger(String fingerName, Point locus, int startRadius, int endRadius, double angle, Duration duration) {\n PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, fingerName);\n Sequence fingerPath = new Sequence(finger, 0);\n\n double midpointRadius = startRadius + (endRadius > startRadius ? 1 : -1) * 20;\n\n // find coordinates for starting point of action (converting from polar coordinates to cartesian)\n int fingerStartx = (int)Math.floor(locus.x + startRadius * Math.cos(angle));\n int fingerStarty = (int)Math.floor(locus.y - startRadius * Math.sin(angle));\n\n // find coordinates for first point that pingers move quickly to\n int fingerMidx = (int)Math.floor(locus.x + (midpointRadius * Math.cos(angle)));\n int fingerMidy = (int)Math.floor(locus.y - (midpointRadius * Math.sin(angle)));\n\n // find coordinates for ending point of action (converting from polar coordinates to cartesian)\n int fingerEndx = (int)Math.floor(locus.x + endRadius * Math.cos(angle));\n int fingerEndy = (int)Math.floor(locus.y - endRadius * Math.sin(angle));\n\n // move finger into start position\n fingerPath.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), fingerStartx, fingerStarty));\n // finger comes down into contact with screen\n fingerPath.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));\n // finger moves a small amount very quickly\n fingerPath.addAction(finger.createPointerMove(Duration.ofMillis(1), PointerInput.Origin.viewport(), fingerMidx, fingerMidy));\n // pause for a little bit\n fingerPath.addAction(new Pause(finger, Duration.ofMillis(100)));\n // finger moves to end position\n fingerPath.addAction(finger.createPointerMove(duration, PointerInput.Origin.viewport(), fingerEndx, fingerEndy));\n // finger lets up, off the screen\n fingerPath.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));\n\n return fingerPath;\n}\n```\n\nFirst we construct a `PointerInput` to represent a finger (as opposed to a mouse pointer) and use it to construct an empty `Sequence` which will hold individual `Actions`. Actions are like steps in the overall movement of a single finger.\n\nI then calculate the coordinates on the screen which we will move the finger between for each action.\n\nNext, we create the actions, adding them to the sequence as we go. We start with a `PointerMove` action to move the finger into its starting position. Then we add a `PointerDown` action, putting our finger in contact with the touchscreen. Now comes the quick initial movement, which we've set to take just one millisecond to complete. We then add a special `Pause` action which waits for 100 milliseconds in our case. Followed by another `PointerMove` action which completes the gesture the rest of the way and takes as much time as was passed into the function as `duration`. Lastly, we add a `PointerUp` action to the sequence to remove our finger from the screen.\n\nBecause I wanted simple \"zoomIn\" and \"zoomOut\" methods and didn't want to specify all these parameters every time I wanted to zoom, I created two more functions which set some defaults.\n\n```java\nprivate Collection zoomIn(Point locus, int distance) {\n return zoom(locus, 200, 200 + distance, 45, Duration.ofMillis(25));\n}\n\nprivate Collection zoomOut(Point locus, int distance) {\n return zoom(locus, 200 + distance, 200, 45, Duration.ofMillis(25));\n}\n```\n\nAll it takes to now reproduce the zooming from the video at the beginning of the article is to call our methods:\n\n```java\n@Test\npublic void ZoomInAndOut() throws InterruptedException {\n // tap center to dismiss toolbars\n WebElement map = driver.findElementById(\"com.google.android.apps.maps:id/mainmap_container\");\n map.click();\n\n Rectangle mapCoordinates = map.getRect();\n Point center = getCenter(mapCoordinates);\n\n driver.perform(zoomOut(center, 450));\n\n Thread.sleep(1000);\n\n driver.perform(zoomIn(center, 450));\n\n Thread.sleep(1000);\n\n driver.perform(zoomOut(center.moveBy(0, 250), 300));\n\n Thread.sleep(1000);\n\n driver.perform(zoomIn(center.moveBy(0, -250), 300));\n\n Thread.sleep(3000);\n}\n```\n\nI originally wrote the methods above because I wanted a generic way to build zoom gestures, but the complexity and trickiness of the Google Maps touch gesture logic resulted in having to build a rather specific set of actions. The timing is very important, but because I used durations if you specify a longer path (on a larger device let's say), the action might then be too fast. At least with the methods written this way, it was easy to experiment with many different combinations of values. A better solution would take the total length the finger has to travel and then calculate a duration based on that.\n\nFeel free to play with actions and experiment! They can get way more complex from here. I'm curious for your solutions, if anyone has written a better set of actions for manipulating this UI.\n\nHere's the test code in its entirety, when it's all put together:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.Point;\nimport org.openqa.selenium.Rectangle;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.interactions.Pause;\nimport org.openqa.selenium.interactions.PointerInput;\nimport org.openqa.selenium.interactions.Sequence;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\nimport java.io.IOException;\nimport java.net.URL;\nimport java.time.Duration;\nimport java.util.Arrays;\nimport java.util.Collection;\n\npublic class Edition067_Zoom_Touch_Gestures {\n\n private AppiumDriver driver;\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"appPackage\", \"com.google.android.apps.maps\");\n caps.setCapability(\"appActivity\", \"com.google.android.maps.MapsActivity\");\n caps.setCapability(\"intentAction\", \"android.intent.action.VIEW\");\n caps.setCapability(\"optionalIntentArguments\", \"-d geo:46.457398,-119.407305\");\n\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n /*\n locus: the center of the touch gesture, the point that fingers are pinching away from or towards. They won't actually touch this point though\n startRadius: distance from center that fingers begin at\n endRadius: distance from center that fingers end at\n pinchAngle: at what angle the fingers pinch around the locus, in degrees. 0 for vertical pinch, 90 for horizontal pinch\n duration: the total amount of time the pinch gesture will take\n */\n private Collection zoom(Point locus, int startRadius, int endRadius, int pinchAngle, Duration duration) {\n // convert degree angle into radians. 0/360 is top (12 O'clock).\n double angle = Math.PI / 2 - (2 * Math.PI / 360 * pinchAngle);\n\n // create the gesture for one finger\n Sequence fingerAPath = zoomSinglefinger(\"fingerA\", locus, startRadius, endRadius, angle, duration);\n\n // flip the angle around to the other side of the locus and get the gesture for the second finger\n angle = angle + Math.PI;\n Sequence fingerBPath = zoomSinglefinger(\"fingerB\", locus, startRadius, endRadius, angle, duration);\n\n return Arrays.asList(fingerAPath, fingerBPath);\n }\n\n /*\n Used by the `zoom` method, for creating one half of a zooming pinch gesture.\n This will return the tough gesture for a single finger, to be put together with\n another finger action to complete the gesture.\n fingerName: name of this input finger for the gesture. Used by automation system to tell inputs apart\n locus: the center of the touch gesture, the point that fingers are pinching away from or towards. They won't actually touch this point though\n startRadius: distance from center that fingers begin at\n endRadius: distance from center that fingers end at\n angle: at what angle the fingers pinch around the locus, in radians.\n duration: the total amount of time the pinch gesture will take\n */\n private Sequence zoomSinglefinger(String fingerName, Point locus, int startRadius, int endRadius, double angle, Duration duration) {\n PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, fingerName);\n Sequence fingerPath = new Sequence(finger, 0);\n\n double midpointRadius = startRadius + (endRadius > startRadius ? 1 : -1) * 20;\n\n // find coordinates for starting point of action (converting from polar coordinates to cartesian)\n int fingerStartx = (int)Math.floor(locus.x + startRadius * Math.cos(angle));\n int fingerStarty = (int)Math.floor(locus.y - startRadius * Math.sin(angle));\n\n // find coordinates for first point that pingers move quickly to\n int fingerMidx = (int)Math.floor(locus.x + (midpointRadius * Math.cos(angle)));\n int fingerMidy = (int)Math.floor(locus.y - (midpointRadius * Math.sin(angle)));\n\n // find coordinates for ending point of action (converting from polar coordinates to cartesian)\n int fingerEndx = (int)Math.floor(locus.x + endRadius * Math.cos(angle));\n int fingerEndy = (int)Math.floor(locus.y - endRadius * Math.sin(angle));\n\n // move finger into start position\n fingerPath.addAction(finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), fingerStartx, fingerStarty));\n // finger comes down into contact with screen\n fingerPath.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));\n // finger moves a small amount very quickly\n fingerPath.addAction(finger.createPointerMove(Duration.ofMillis(1), PointerInput.Origin.viewport(), fingerMidx, fingerMidy));\n // pause for a little bit\n fingerPath.addAction(new Pause(finger, Duration.ofMillis(100)));\n // finger moves to end position\n fingerPath.addAction(finger.createPointerMove(duration, PointerInput.Origin.viewport(), fingerEndx, fingerEndy));\n // finger lets up, off the screen\n fingerPath.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));\n\n return fingerPath;\n }\n\n /*\n Simplified method for zooming in.\n Defaults to a 45 degree angle for the pinch gesture.\n Defaults to a duration of half a second\n Fingers start 50px from locus\n\n locus: the center of the pinch action, fingers move away from here\n distance: how far fingers move outwards, starting 100px from the locus\n */\n private Collection zoomIn(Point locus, int distance) {\n return zoom(locus, 200, 200 + distance, 45, Duration.ofMillis(25));\n }\n\n /*\n Simplified method for zooming out.\n Defaults to a 45 degree angle for the pinch gesture.\n Defaults to a duration of half a second\n Fingers finish 50px from locus\n\n locus: the center of the pinch action, fingers move towards here\n distance: how far fingers move inwards, they will end 100px from the locus\n */\n private Collection zoomOut(Point locus, int distance) {\n return zoom(locus, 200 + distance, 200, 45, Duration.ofMillis(25));\n }\n\n @Test\n public void ZoomInAndOut() throws InterruptedException {\n // tap center to dismiss toolbars\n WebElement map = driver.findElementById(\"com.google.android.apps.maps:id/mainmap_container\");\n map.click();\n\n Rectangle mapCoordinates = map.getRect();\n Point center = getCenter(mapCoordinates);\n\n driver.perform(zoomOut(center, 450));\n\n Thread.sleep(1000);\n\n driver.perform(zoomIn(center, 450));\n\n Thread.sleep(1000);\n\n driver.perform(zoomOut(center.moveBy(0, 250), 300));\n\n Thread.sleep(1000);\n\n driver.perform(zoomIn(center.moveBy(0, -250), 300));\n\n Thread.sleep(3000);\n }\n\n private Point getCenter(Rectangle rect) {\n return new Point(rect.x + rect.getWidth() / 2, rect.y + rect.getHeight() / 2);\n }\n}\n```\n\nThat's all for this week. The example can be found with all our example code [on Github](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition067_Zoom_Touch_Gestures.java).\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Generating Touch Gestures to Zoom In and Out on Google Maps","shortDesc":"Google Maps requires some creative touch gestures to zoom in and out, providing a good demonstration of Appium's versatile gesture API","liveAt":"2019-05-01 10:00","canonicalRef":"https://www.headspin.io/blog/generating-touch-gestures-to-zoom-in-and-out-on-google-maps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/67-generating-touch-gestures-to-zoom-in-and-out-on-google-maps","slug":"67-generating-touch-gestures-to-zoom-in-and-out-on-google-maps","path":"/editions/67-generating-touch-gestures-to-zoom-in-and-out-on-google-maps","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0066.md","num":66,"rawMd":"\n\nAppium has long had the ability to temporarily close the current app under test and bring it back (the process of backgrounding an app), since testing the ability of an app to recover from being closed is important to app developers. When the XCUITest framework was released and we built the XCUITest driver for Appium, we discovered that once the app under test was backgrounded we could still launch other apps and automate them. This includes the built-in iOS apps like Messages, and the Settings app!\n\nInitially, I thought that this was unintentional and that the capability would be removed, but it has stuck around for a couple years and some Appium features now depend on it.\n\nWhile new to iOS, Android has long allowed testers to automate apps outside of the app under test.\n\nIf you noticed from the [last couple articles about capturing network requests](https://appiumpro.com/editions/65), we used the `mobile: installCertificate` command to install a custom SSL certificate. When given this command, Appium opens up the Settings app and navigates through menus, to install the certificate. This is achieved through the same automation Appium applies to your app, and you can easily do the same.\n\nAll it takes it launching an app using its bundleId, while your test is running:\n\n```java\n// open Settings app\ndriver.activateApp(\"com.apple.Preferences\");\n```\n\nThis will launch the Settings app, and you can now inspect and automate as normal. To go back to the app under test, launch it the same way:\n\n```java\ndriver.activateApp(\"your.bundle.id\");\n```\n\nIf you don't know your app's bundleId there are numerous ways to get it. Appium needs it to automate your app in the first place, so I just took a look at the Appium logs for one of my test sessions and it included this line in the initial session setup:\n```\n[debug] [iOS] Getting bundle ID from app '/var/folders/mj/pgmgyh_95rsbs10yl46b73zc0000gn/T/2019319-8020-1rqx9az.mn5jj/TheApp.app': 'io.cloudgrey.the-app'\n```\n\nOn Android, the process is the same, but you use the appPackage or appActivity instead of the bundleId.\n\nThat's it. I still don't expect this to work forever on iOS, but might as well use it while we have it. Your mileage may vary, depending on the app. The Settings app is constructed very simply, but more complex apps may exhibit strange behavior when attempting to automate outside the app under test.\n\nThe Settings apps have been the most useful for us to automate, anything a user can change about their phone, we can change too.\n\nAnother app you could automate would be the Messages app, with the bundleId `com.apple.MobileSMS`. You could use this for some tricky tests involving communication via SMS outside of your app, but if all you want to do is get a verification text, it may be better to use one of the many services which provide temporary phone numbers you can access programatically.\n\nI found [this list](https://github.com/joeblau/apple-bundle-identifiers) of the bundleIds of iOS version 12 system apps, [this list](https://docs.google.com/spreadsheets/d/1jgJewUJma2rCfOdphGqfwieBRNHQd1FRJ1q0F6N_SdM/pub?hl=en_US&hl=en_US&hl=en_US&single=true&gid=0&output=html) of Android system apps. Additionally, we outlined how to find the package names of all apps on your Android device in (this earlier article about Android activities and intents)[https://appiumpro.com/editions/56].\n\nHere's a quick example demonstrating the launch of system apps on iOS:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport org.junit.After;\nimport org.junit.Test;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\nimport java.net.MalformedURLException;\nimport java.net.URL;\n\n\npublic class Edition066_Automating_iOS_System_Apps {\n\n private String IOS_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip\";\n\n private AppiumDriver driver;\n\n @After\n public void Quit() {\n driver.quit();\n }\n\n @Test\n public void launchSystemApp() throws MalformedURLException, InterruptedException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.2\");\n caps.setCapability(\"deviceName\", \"iPhone Xs\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n caps.setCapability(\"app\", IOS_APP);\n\n driver = new IOSDriver(new URL(\"http://0.0.0.0:4723/wd/hub\"), caps);\n\n WebElement picker = driver.findElementByAccessibilityId(\"Picker Demo\");\n picker.click();\n\n // open iMessage\n driver.activateApp(\"com.apple.MobileSMS\");\n\n // do things with SMS here\n\n Thread.sleep(3000);\n\n // open Settings app\n driver.activateApp(\"com.apple.Preferences\");\n\n // regular automation commands to change device settings here\n\n // go back to our app\n driver.activateApp(\"io.cloudgrey.the-app\");\n\n Thread.sleep(3000);\n }\n}\n```\n\nThat's all for this week. The example can be found with all our example code [on Github](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition065_Capture_Network_Requests.java).\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Automating System Apps with Appium","shortDesc":"Built-in apps which come preinstalled on devices can be automated by Appium on both iOS and Android. This is especially useful when it comes to automating the settings apps for special test requirements.","liveAt":"2019-04-24 10:00","canonicalRef":"https://www.headspin.io/blog/automating-system-apps-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/66-automating-system-apps-with-appium","slug":"66-automating-system-apps-with-appium","path":"/editions/66-automating-system-apps-with-appium","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0065.md","num":65,"rawMd":"\n\nWrapping up a short series on capturing device network traffic, I'm happy to present [`mitmproxy-java`](https://github.com/appium/mitmproxy-java), a small library which allows convenient access to network requests of devices made in the middle of your test runs. It has the following features which I was unable to find using other methods in Java:\n - Starts the proxy server as a background task.\n - Allows for passing in a lambda function which gets called for every intercepted message, allowing you to manage the recording of data any way you see fit.\n - Allows for modifying the responses from the proxy, so you can inject arbitrary data and states into your app.\n - Captures HTTPS traffic even when ip addresses are used instead of host names.\n\nThe first three advantages come from the wrapper code in `mitmproxy-java` which is basically a Java version of the great Node.js module I found for the same purpose: [mitmproxy-node](https://github.com/jvilk/mitmproxy-node). The last bullet point comes from the use of `mitmproxy`.\n\nTraditionally, the testing community mostly seems to use Browsermob Proxy, but I found it has not been maintained recently and can't support Android emulators due to the issue with HTTPS traffic and ip addresses. I'm hoping that people will be able to find `mitmproxy-java` as a suitable upgrade.\n\nBut please help! I put a lot of work into it but I'm not a Java expert. The way I currently handle exceptions isn't friendly. Hop onto github and submit pull requests or make issues if you run into trouble. If the community is supportive, we can improve it further.\n\nOh, this should work for Selenium too, if you set up the browsers to proxy correctly.\n\n## Setup ##\n\nFor those just tuning in, see the past two articles on capturing network traffic to learn about what we're doing and how it works:\n - [Capturing iOS Simulator Network Traffic with Appium](https://appiumpro.com/editions/62)\n - [Capturing Android Emulator Network Traffic with Appium](https://appiumpro.com/editions/63)\n\nThose two articles also go through the setup needed for configuring devices, this post will focus on setting up `mitmproxy-java` and how to write the Java test code.\n\nWhile `mitmproxy-java` will start the proxy server for us programmatically, we need to install `mitmproxy` ourselves, just like we did in the previous articles. Make sure to install with `pip3` since installing with other methods, misses some python dependencies which we need.\n\n```\nsudo pip3 install mitmproxy\n```\n\nWhen running `mitmproxy-java`, we need to supply it with the location of the `mitmdump` executable. `mitmdump` is installed automatically when you install `mitmproxy` and is a commandline version of `mitmproxy` which isn't interactive and runs in the background. Let's get that location and make a note of it for later.\n\n```\nwhich mitmdump\n```\n\nFor me the output is `/usr/local/bin/mitmdump`.\n\nNext, we need to install the Python websockets module. the way `mitmproxy-java` works, is it starts `mitmdump` with a special Python plugin which is included inside the `mitmproxy-java` jar. This plugin runs inside `mitmdump` and connects to a websocket server hosted by `mitmproxy-java`. The Python code then transfers request/response data to the Java code over the websocket.\n\n```\npip3 install websockets\n```\n\nThat should be all the setup we need on our host machine, now on to the actual test code.\n\n## Writing a Test Using mitmproxy-java ##\n\nInclude the `mitmproxy-java` jar in your project. The library is [hosted on Maven Central Repository](https://mvnrepository.com/artifact/io.appium/mitmproxy-java/1.6.1).\n\nAdd the following to your pom file:\n\n```xml\n\n io.appium\n mitmproxy-java\n 1.6.1\n\n```\n\nOr in your `build.gradle` file, for Gradle users:\n\n```groovy\ncompile group: 'io.appium', name: 'mitmproxy-java', version: '1.6.1'\n```\n\nYou can now access two classes in your test code:\n`MitmproxyJava` - The class which starts and stops the proxy.\n`InterceptedMessage` - A class used to represent messages intercepted by the proxy. A \"message\" includes both an HTTP request, and its matching response.\n\nThe constructor for `MitmproxyJava` takes two arguments. The first is a String with the path to the `mitmdump` executable on your computer. We got this value earlier in the setup section. The second argument is a lambda function which the `MitmproxyJava` instance will call every time it intercepts a network request. You can do anything you like with the `InterceptedMessage` passed in.\nIn the following example, we create a `List` of `InterceptedMessage` objects and instantiate a new `MitmproxyJava` instance. every intercepted message gets added to our list, which is in scope for the rest of the test.\n\n```java\nList messages = new ArrayList();\n\n// remember to set local OS proxy settings in the Network Preferences\nproxy = new MitmproxyJava(\"/usr/local/bin/mitmdump\", (InterceptedMessage m) -> {\n System.out.println(\"intercepted request for \" + m.requestURL.toString());\n messages.add(m);\n return m;\n});\n```\n\nNotice that we return the message from the lambda function. If we forget to return it, no worries, this is the implicit behavior. If you block or throw an error though, then the message response never completes its journey to your test device.\n\nYou can also modify the response in the `InterceptedMessage`. Modifying `m.responseHeaders` and setting different bytes in the content of `m.responseBody` will result in overwriting the data which the device receives in response to its request.\n\nNow that we've instantiated our `MitmproxyJava` object, all we need to do is call\n```java\nproxy.start();\n```\nto start the proxy server and start collecting responses. This method call runs in a separate thread. Call\n```java\nproxy.stop();\n```\nto shut down.\n\nThe proxy, by default, runs on `localhost:8080` just like in the examples from the previous articles. One future feature should be to allow configuration of this port.\n\nThat's it!\n\nHere's an example of an entire test for Android and iOS, using `mitmproxy-java`:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport org.junit.After;\nimport org.junit.Test;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\nimport java.io.IOException;\nimport java.net.URISyntaxException;\nimport java.net.URL;\nimport java.nio.file.Files;\nimport java.nio.file.Path;\nimport java.nio.file.Paths;\nimport java.util.*;\nimport java.util.concurrent.ExecutionException;\nimport java.util.concurrent.TimeoutException;\n\nimport static junit.framework.TestCase.assertTrue;\n\n\npublic class Edition065_Capture_Network_Requests {\n\n private String ANDROID_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.9.0.apk\";\n private String IOS_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.6.1/TheApp-v1.6.1.app.zip\"; // in order to download, you may need to install the mitmproxy certificate on your operating system first. Or download the app and replace this capability with the path to your app.\n\n private AppiumDriver driver;\n private MitmproxyJava proxy;\n\n @After\n public void Quit() throws IOException, InterruptedException {\n proxy.stop();\n driver.quit();\n }\n\n @Test\n public void captureIosSimulatorTraffic() throws IOException, URISyntaxException, InterruptedException, ExecutionException, TimeoutException {\n List messages = new ArrayList();\n\n // remember to set local OS proxy settings in the Network Preferences\n proxy = new MitmproxyJava(\"/usr/local/bin/mitmdump\", (InterceptedMessage m) -> {\n System.out.println(\"intercepted request for \" + m.requestURL.toString());\n messages.add(m);\n return m;\n });\n\n proxy.start();\n\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.0\");\n caps.setCapability(\"deviceName\", \"iPhone Xs\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n caps.setCapability(\"app\", IOS_APP);\n\n driver = new IOSDriver(new URL(\"http://0.0.0.0:4723/wd/hub\"), caps);\n\n // automatically install mitmproxy certificate. Can be skipped if done manually on the simulator already.\n Path certificatePath = Paths.get(System.getProperty(\"user.home\"), \".mitmproxy\", \"mitmproxy-ca-cert.pem\");\n Map args = new HashMap<>();\n byte[] byteContent = Files.readAllBytes(certificatePath);\n args.put(\"content\", Base64.getEncoder().encodeToString(byteContent));\n driver.executeScript(\"mobile: installCertificate\", args);\n\n WebElement picker = driver.findElementByAccessibilityId(\"Picker Demo\");\n picker.click();\n WebElement button = driver.findElementByAccessibilityId(\"learnMore\");\n button.click();\n WebDriverWait wait = new WebDriverWait(driver, 5);\n wait.until(ExpectedConditions.alertIsPresent());\n driver.switchTo().alert().accept();\n\n\n assertTrue(messages.size() > 0);\n\n InterceptedMessage appiumIORequest = messages.stream().filter((m) -> m.requestURL.getHost().equals(\"history.muffinlabs.com\")).findFirst().get();\n\n assertTrue(appiumIORequest.responseCode == 200);\n }\n\n @Test\n public void captureAndroidEmulatorTraffic() throws IOException, URISyntaxException, InterruptedException, ExecutionException, TimeoutException {\n List messages = new ArrayList();\n\n // remember to set local OS proxy settings in the Network Preferences\n proxy = new MitmproxyJava(\"/usr/local/bin/mitmdump\", (InterceptedMessage m) -> {\n System.out.println(\"intercepted request for \" + m.requestURL.toString());\n messages.add(m);\n return m;\n });\n\n proxy.start();\n\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"platformVersion\", \"9\");\n caps.setCapability(\"deviceName\", \"test-proxy\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"app\", ANDROID_APP);\n\n driver = new AndroidDriver(new URL(\"http://0.0.0.0:4723/wd/hub\"), caps);\n\n WebElement picker = driver.findElementByAccessibilityId(\"Picker Demo\");\n picker.click();\n WebElement button = driver.findElementByAccessibilityId(\"learnMore\");\n button.click();\n WebDriverWait wait = new WebDriverWait(driver, 5);\n wait.until(ExpectedConditions.alertIsPresent());\n driver.switchTo().alert().accept();\n\n\n assertTrue(messages.size() > 0);\n\n InterceptedMessage appiumIORequest = messages.stream().filter((m) -> m.requestURL.getPath().equals(\"/date/1/1\")).findFirst().get();\n\n assertTrue(appiumIORequest.responseCode == 200);\n }\n}\n```\n\nFull source code for this example can be found with all our example code [on Github](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition065_Capture_Network_Requests.java).\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Capturing Network Traffic in Java with Appium","shortDesc":"Previous articles went over how to capture network traffic on iOS Simulators and Android Emulators during Appium tests, but all the sample code was in Javascript. This edition introduces `mitmproxy-java`, a library based off of `mitmproxy-node`, which allows one to easily run a proxy and capture network requests in a background process in Java.","sponsor":"headspin","sponsorLink":"https://www.headspin.io/solutions/performance-optimization?utm_source=appiumpro&utm_medium=referral&utm_campaign=pilot","sponsorBlurb":"HeadSpin's packet engine does network capture on real devices and cell network radios, and delivers protocol analysis, PCAP, traceroute, and root cause analysis of common network issues that affect performance.","liveAt":"2019-04-17 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-network-traffic-in-java-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/65-capturing-network-traffic-in-java-with-appium","slug":"65-capturing-network-traffic-in-java-with-appium","path":"/editions/65-capturing-network-traffic-in-java-with-appium","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0064.md","num":64,"rawMd":"\n\nIf you've used Android apps for any length of time, you've no doubt noticed these little notifications that pop up and fade away with time:\n\n

\n \"Hello,\n

\n\nThese are called [toast messages](https://developer.android.com/guide/topics/ui/notifiers/toasts), and are an important tool for Android app designers, because they don't steal focus from the current activity. Your app might complete a background task while the user is playing a game, and with toasts you are able to convey this information without taking the user away from their present context.\n\nOf course, toast messages can prove a challenge for automation, not just because of their ephemeral nature. From the perspective of the Android Accessibility layer, toast messages aren't visible! If you try to get the XML source from an Appium session while a toast is present on screen, you won't find its text anywhere. Luckily, with the advent of the [Espresso driver](https://github.com/appium/appium-espresso-driver/), we have the ability to match text against on-screen toasts! Let's see how it all works.\n\nFirst, we need a way to actually produce toast messages we can use for testing. I could add some behavior in my app that produces toasts, like a real app would, but instead I'm going to rely on another cool feature of the Espresso driver we've covered in the past -- [calling app-internal methods](https://appiumpro.com/editions/51). Since I've already got the plumbing hooked up in The App, I can just write myself a handy little helper method that will display toasts for me from my test code:\n\n```java\nprivate void raiseToast(String text) {\n\tImmutableMap scriptArgs = ImmutableMap.of(\n\t\t\"target\", \"application\",\n\t\t\"methods\", Arrays.asList(ImmutableMap.of(\n\t\t\t\"name\", \"raiseToast\",\n\t\t\t\"args\", Arrays.asList(ImmutableMap.of(\n\t\t\t\t\"value\", text,\n\t\t\t\t\"type\", \"String\"\n\t\t\t))\n\t\t))\n\t);\n\n\tdriver.executeScript(\"mobile: backdoor\", scriptArgs);\n}\n```\n\n(Of course, in a real testing scenario, the toasts would be generated as a result of some app behavior). Once I've got toasts showing up, I need a way to check what they say for verification. Unfortunately, we don't have a method for getting the text from a toast message. What we have instead is a method which takes a string and tells us whether the on-screen toast matches that string or not. This is enough for our purposes of verification. So let's check out how to use the `mobile: isToastVisible` method:\n\n```java\nImmutableMap args = ImmutableMap.of(\n\t\"text\", \"toast text to match\",\n\t\"isRegexp\", false\n);\ndriver.executeScript(\"mobile: isToastVisible\", args);\n```\n\nLike all `mobile:` methods, we first need to construct a map of our arguments. This method takes two parameters: the text we want to look for, and a flag which tells Appium whether this text is in the form of a bare string or a regular expression. If we set `isRegexp` to `true`, then we can look for toast messages using more advanced criteria, limited only by what we can express in a regular expression. Finally, we call `executeScript` as the way of accessing the `mobile:` method.\n\nThis is great, but as we've mentioned already, toasts are a time-sensitive phenomenon. So we probably want to start looking for a matching toast before it pops up, so we're sure we don't miss it. To this end, we can use a custom Explicit Wait. It's possible to use the Java client's `ExpectedCondition` interface to define our own custom expected conditions, so that's what we'll do. Here's a helper method that defines a new `ExpectedCondition` called `toastMatches`:\n\n```java\npublic static ExpectedCondition toastMatches(String matchText, Boolean isRegexp) {\n\treturn new ExpectedCondition() {\n\t\t@Override\n\t\tpublic Boolean apply(WebDriver driver) {\n\t\t\tImmutableMap args = ImmutableMap.of(\n\t\t\t\t\"text\", matchText,\n\t\t\t\t\"isRegexp\", isRegexp\n\t\t\t);\n\t\t\treturn (Boolean) ((JavascriptExecutor)driver).executeScript(\"mobile: isToastVisible\", args);\n\t\t}\n\n\t\t@Override\n\t\tpublic String toString() {\n\t\t\treturn \"toast to be present\";\n\t\t}\n\t};\n}\n```\n\nAll we do is override the appropriate methods of the `ExpectedCondition` class, and ensure we have appropriate typing in a few places, and we've got ourselves a nice self-contained way of waiting for toast messages, in conjunction with (for example) `WebDriverWait`. Since all the pieces are now in places, let's take a look at what our test method itself could look like:\n\n```java\n@Test\npublic void testToast() {\n\tWebDriverWait wait = new WebDriverWait(driver, 10);\n\n\tfinal String toastText = \"Catch me if you can!\";\n\traiseToast(toastText);\n\n\twait.until(toastMatches(toastText, false));\n\n\traiseToast(toastText);\n\twait.until(toastMatches(\"^Catch.+!\", true));\n}\n```\n\nIn this test, we define a `WebDriverWait` and use it with our `toastMatches` condition. You can see that we perform a match with both available modes, first by matching the exact toast string, and secondly by using a regular expression, highlighting how we could verify the presence of a valid toast message even if it contains dynamically generated content.\n\nThat's it! If you haven't checked out Appium's Espresso driver for Android, validating toast messages is a good reason to give it a try. And if you want to see a working example with all the boilerplate, you can find it [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition064_Android_Toast.java) as always.\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Validating Android Toast Messages","shortDesc":"A common app notification method on the Android platform utilizes 'toast' messages, which are displayed for a brief period of time and then disappear. Using methods available in Appium's Espresso driver, in conjunction with Explicit Waits, it's possible to verify the behavior of toast messages within your app.","liveAt":"2019-04-10 10:00","canonicalRef":"https://www.headspin.io/blog/validating-android-toast-messages"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/64-validating-android-toast-messages","slug":"64-validating-android-toast-messages","path":"/editions/64-validating-android-toast-messages","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0063.md","num":63,"rawMd":"\n\nLast week, we covered [how to capture network traffic generated by an iOS simulator](https://appiumpro.com/editions/61) within an Appium test. This week, we'll do the same thing with Android Emulators.\n\nA brief recap from last week: We can start a man-in-the-middle proxy called `mitmproxy` which will capture all the network traffic from the emulator, and let us access the requests and responses from our test script. We can then perform advanced assertions in our test, making sure that the app is sending requests to other services as expected. We can also modify the responses, forcing the app into states we dictate, for testing purposes. For more details on proxies and why I chose mitmproxy, read the previous article.\n\nAs with last week, the example code is written in Javascript. I hope to announce a convenient way to capture network traffic with mitmproxy from Java code in a future article.\n\n### Setting up mitmproxy for Android emulators ###\n\nMitmproxy can be installed and run on Windows from within the Windows Subsystem for Linux (WSL). These instructions are for MacOS, but should work the same from WSL.\n\nInstall mitmproxy. You may need to make sure you have Python3 and `pip3` installed first. Installing this way is required for the Javascript library we will be using later.\n```\nsudo pip3 install mitmproxy\n```\n\nAnd then run it:\n```\nmitmproxy\n```\n\nThe proxy runs on localhost port `8080` by default. The empty pane it displays should fill up with network traffic once we configure our simulator to point to it.\n\nAndroid emulators have a built-in setting for configuring HTTP and HTTPS proxies. Start an emulator and open the emulator settings pane (not the Settings app inside the emulator). Click on the `Settings` tab on the left side, then the inset `Proxy` tab on top.\n\n

\n \"Android\n

\n\nSet a \"Manual Proxy Configuration\" with host name `http://0.0.0.0` and port `8080`.\n\n

\n \"Android\n

\n\nNow the emulator will try to forward all traffic through mitmproxy which we are running on the same port. To confirm that things are set up so far, open a web browser on the emulator and try to load `https://appiumpro.com`, the page should not load!\n\nThe reason it does not load is because mitmproxy is intercepting the request and forwarding the response, but this site uses HTTPS and mitmproxy signs each response with the special mitmproxy certificate. By default, our emulator does not trust the mitmproxy certificate, and refuses to load the page. The next step is to install and trust the mitmproxy certificate.\n\nThis is where things get a little tricky. Most of the guides you'll find online for how to set up a proxy for Android emulators will only work on Android version 6 and lower. Android version 7, \"Nougat\", introduced stricter security rules when it comes to installing SSL certificates. If you were to navigate to the special url hosted by mitmproxy: `mitm.it` and follow the instructions there for installing and trusting the mitmproxy certificate, this would only fix the web browsers on the device. You could load `https://appiumpro.com` in the browser, but HTTPS requests which originate from inside native apps would still be blocked. This is because the device keeps two lists of trusted certificates: user certificates and system certificates. Installing a certificate through the UI puts it into the user certificate list, but only certificates in the system list are trusted for app-based network requests.\n\nSo now we need to add the mitmproxy certificate to the system certificate list. This can only be done with root access to the device, which we have with our emulators. This process should also work on real devices that have been rooted, but you may need to find a different set of commands which works on your particular device. These instructions should work on most newer emulators.\n\nA silver lining to this process is that installing a user certificate requires enabling the device's lock screen but this method of installing a system certificate allows us to keep the lock screen disabled.\n\nFrom the commandline, perform the following:\n\nFirst, launch the emulator with the `-writable-system` flag. This will allow us to write to the system files of the device. My AVD is named `test-proxy`. You'll have to insert the name of your emulator here.\n\n```bash\nemulator -avd test-proxy -writable-system &\n```\n\nThat should boot up the emulator. Once booted, we unlock the device further:\n\n```bash\nadb root\nadb remount\n```\n\nLet's now find our certificate file and push it to the device. The first time we ran mitmproxy, it saved its certificates to the home directory. Let's save that to a variable to use later.\n\n```bash\nca=~/.mitmproxy/mitmproxy-ca-cert.pem\n```\n\nWe also need to get a special hash to identify the certificate. The filename of the certificate on the emulator needs to be set as the hash of the certificate.\n```bash\nhash=$(openssl x509 -noout -subject_hash_old -in $ca)\n```\n\nIf you don't have `openssl` installed on your computer, you can cheat and just use the value `c8750f0d`. The mitmproxy certificate is the same for everybody, so unless they change it in a later version, this should be fine.\n```bash\nhash=c8750f0d\n```\n\nOk, now we install it on the device:\n```bash\nadb push $ca /system/etc/security/cacerts/$hash.0\n```\n\nAnd let's take adb back out of root mode, because I've found that the emulators tend to crash when restarting or lose the mitmproxy certificate unless you take this step:\n```bash\nadb unroot\n```\n\nThat's it. `https://appiumpro.com` should load successfully on the device's browser with mitmproxy running. Requests made by native apps will be logged by the proxy as well. Take a look at the terminal where `mitmproxy` is running and you should see requests being logged there when you perform actions on the emulator (arrow keys scroll the mitmproxy UI).\n\nJust to double check ourselves, we can locate the mitmproxy certificate in the list of system-trusted certificates. On the device, open the settings app, select \"Security & Location\"->Advanced->\"Encryption and Credentials\"->\"Trusted Credentials\". Scroll down way past all the default certificates and find one named \"mitmproxy\" (they're in alphabetical order). Note the tabs at the top which differentiate between the \"system\" certificates and the \"user\" certificates.\n\n

\n \"Android\n

\n\nIf we see it there, we're ready to run our test scripts.\n\n### Automating Certificate Installation ###\n\nFor new emulators, the steps above need to be repeated. If you want to automate setting the device proxy, this can be done by adding a `-http-proxy` argument to the command for starting the emulator:\n```bash\nemulator -avd test-proxy -http-proxy http://0.0.0.0:8082\n```\nI also found that if you're really stuck, you can add the `-debug-proxy` flag and the emulator will log information about the proxy it is using:\n```bash\nemulator -avd test-proxy -debug-proxy -http-proxy http://0.0.0.0:8082\n```\n\nThere is a [pull request](https://github.com/appium/appium-android-driver/pull/386) in progress to add a new desired capability to Appium which would automatically install the certificate to the device. Then most the steps here could be skipped!\n\nAppium already has the `avdArgs` desired capability, which can be used to pass the `-http-proxy` setting when starting the emulator.\n\n### Capturing Traffic Within a Node.js Test Script ###\n\nThe example code for capturing Android emulator network traffic is almost identical to the iOS example. The only difference besides the necessary changes to desired capabilities, is that Android emulators seem to send requests directly to an IP address rather than a host name. Our App's [code](https://github.com/cloudgrey-io/the-app/blob/d32513aa269429099d0b9ede296e617d948d7251/screens/PickerScreen.js#L41) makes a request to `https://history.muffinlabs.com/date/1/1` but mitmproxy captures a packet with the address: `https://69.163.152.183/date/1/1`. This is valid behavior but other man-in-the-middle proxies (such as BrowserMob Proxy) cannot handle traffic like this.\n\nLet's insert code to start mitmproxy before our test, then filter and access the captured traffic so we can make assertions on it.\n\nThere are a couple different approaches to this. What I really want is the ability to programmatically start mitmproxy and access the requests within my test. Mitmproxy is designed with a nicely-featured addon framework which supports plugins written in Python. The way the designers expects you to run mitmproxy is as a standalone service, and pass python script files to it using the `-s` commandline flag. We could write a python addon and start/stop the proxy as a separate shell process, but I want to avoid that if I can.\n\nInstead, I found a [great little Node.js module](https://github.com/jvilk/mitmproxy-node) which starts mitmproxy as a separate process and loads a special Python addon which opens a websocket connection so that we can get information in our Javascript code while the proxy is running.\n\nThe following test opens our usual demo app, [TheApp](https://github.com/cloudgrey-io/the-app/releases), opens our pickerwheel demo, and taps the button. This action results in the app sending a request to `history.muffinlabs.com` which returns some historical events for the app to display.\n\nThe test starts mitmproxy and logs all requests sent by the app to an array. At the end of the test, we verify that the request we expected the app to send was indeed sent.\n\n```javascript\nlet test = require('ava')\nlet fs = require('fs-extra')\nlet os = require('os')\nlet path = require('path')\nlet { remote } = require('webdriverio')\nlet Mitmproxy = require('mitmproxy').default\n\nlet proxy, driver\nlet interceptedMessages = []\n\n// this function will get called every time the proxy intercepts a request\nlet requestHandler = (message) => {\n let req = message.request\n console.log('************************************')\n console.log('mitmproxy intercepted a request')\n console.log(req.method)\n console.log(req.rawUrl)\n console.log(message.requestBody.toString())\n console.log('************************************')\n interceptedMessages.push(message)\n}\n\ntest.before(async t => {\n // start mitmproxy\n proxy = await Mitmproxy.Create(requestHandler, [], true, true)\n\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'Android',\n platformVersion: '9',\n deviceName: 'test-proxy',\n automationName: 'UiAutomator2',\n app: '/Users/jonahss/Workspace/TheApp-v1.9.0.apk', //'https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.apk',\n },\n logLevel: 'silent'\n })\n})\n\ntest('getting the event for a day, via request to history.muffinlabs.com', async t => {\n let pickerDemo = await driver.$('~Picker Demo')\n await pickerDemo.click()\n let button = await driver.$('~learnMore')\n await button.click()\n // wait for alert\n let alertIsPresent = async () => {\n try { return await driver.getAlertText(); } catch { return false; }\n }\n await driver.waitUntil(alertIsPresent, 4000)\n await driver.dismissAlert()\n\n t.true(interceptedMessages.length > 0)\n t.true(interceptedMessages.some(m => /https:\\/\\/.*\\/date\\/1\\/1/.test(m.request.rawUrl)))\n})\n\ntest.after.always(async t => {\n t.log('shutting down')\n await proxy.shutdown()\n await driver.deleteSession()\n})\n```\n\n### Modifying Network Responses Recieved By The Device ###\n\nMitmproxy allows us to get even more clever. Just changing a couple lines in the test above, we can intercept the app's request to `history.muffinlabs.com` and rewrite the contents of the response to contain the data of our choosing.\n\nThis can be useful in getting the app into a predictable state. The current code chooses a random historical event to display every time, but we can respond with a single event of our choosing and force the app to display what we want. We can also force the UI to display specific values, so we can test long strings, negative numbers, error states, etc, without having to actually set up the necessary context on the backend. All this can be done from the test script!\n\n```javascript\nlet test = require('ava')\nlet { remote } = require('webdriverio')\nlet Mitmproxy = require('mitmproxy').default\n\nlet proxy, driver\nlet interceptedMessages = []\n\n// this is the response we will return from our proxy, instead of what the site usually returns\nlet injectedResponse = {\n date: \"January 1\",\n url: \"https://wikipedia.org/wiki/January_1\",\n data: {\n Events: [\n {\n year: \"2019\",\n text: \"Tests Passed\",\n }\n ]\n }\n}\n\nlet requestHandler = (message) => {\n message.setResponseBody(Buffer.from(JSON.stringify(injectedResponse), 'utf8'))\n}\n\ntest.before(async t => {\n proxy = await Mitmproxy.Create(requestHandler, [], true, true)\n\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'Android',\n platformVersion: '9',\n deviceName: 'test-proxy',\n automationName: 'UiAutomator2',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.apk',\n },\n logLevel: 'silent'\n })\n})\n\ntest(`insert our own event for a day, assert that it's displayed`, async t => {\n let pickerDemo = await driver.$('~Picker Demo')\n await pickerDemo.click()\n let button = await driver.$('~learnMore')\n await button.click()\n\n // wait for alert\n let alertIsPresent = async () => {\n try { return await driver.getAlertText(); } catch { return false; }\n }\n await driver.waitUntil(alertIsPresent, 4000)\n\n let alertText = await driver.getAlertText()\n await driver.dismissAlert()\n\n // assert that the alertText is the same as the packet we injected\n t.true(/Tests Passed/.test(alertText))\n})\n\ntest.after.always(async t => {\n t.log('shutting down')\n await proxy.shutdown()\n await driver.deleteSession()\n})\n```\n\nThat's all for this week. [Sample code is on github, as usual](https://github.com/cloudgrey-io/appiumpro/tree/master/javascript).\n","metadata":{"lang":"all","platform":"Android","subPlatform":"All Devices","title":"Capturing Android Emulator Network Traffic with Appium","shortDesc":"Using a Proxy, we can capture the exact network packets sent from an Android emulator and make assertions on their contents or timing. We can even modify the responses at will!","sponsor":"headspin","sponsorLink":"https://www.headspin.io/solutions/performance-optimization?utm_source=appiumpro&utm_medium=referral&utm_campaign=pilot","sponsorBlurb":"HeadSpin's packet engine does network capture on real devices and cell network radios, and delivers protocol analysis, PCAP, traceroute, and root cause analysis of common network issues that affect performance.","liveAt":"2019-04-03 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-android-emulator-network-traffic-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/63-capturing-android-emulator-network-traffic-with-appium","slug":"63-capturing-android-emulator-network-traffic-with-appium","path":"/editions/63-capturing-android-emulator-network-traffic-with-appium","news":null,"tags":["all","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0062.md","num":62,"rawMd":"\n\nThe next couple editions of Appium Pro will be about capturing network requests made by devices during a test. If we can access network requests from our test code, we could make assertions based on specific requests being sent to our server, or assert that the data in network responses are displayed in the UI. We can also capture network data to help debug issues when tests fail. We could also make assertions on the performance of our network calls, by checking the timing between requests and responses.\n\nIt's relatively easy to get network logs from Chrome and Safari sessions, since the browser debug ports make that information available, but what if we want to capture requests made by native apps?\n\nThe solution is to use a man-in-the-middle proxy. A proxy of this sort is a program which we insert between the device and the internet. We tell the device to route all requests through our proxy, which records the traffic as it passes through. Not only does it record it, but the proxy we will be using can also be configured to _modify_ the traffic. This opens up more potential uses in testing: we can test the failure cases when certain urls can't be reached, and we can modify responses to suit our needs for consistent data.\n\nMan-in-the-middle proxies also do a lot of tricky work breaking the security built into networking systems. Capturing someone's encrypted HTTPS traffic is what malicious attackers would want to do. We are but humble testers, but how is the system to know that? The proxy does a lot of work to give us fake authority to decode encrypted network traffic, once we perform some initial setup.\n\nWith a proxy we can even _modify_ the network responses our device receives, allowing us to test with less random values. We can also use this technique to force the app into a particular state we want to test. All from within our test code, involving no changes to the app.\n\nFor these articles, we will use a proxy called [mitmproxy](https://mitmproxy.org/). Much of the web testing and Selenium world is used to using Browsermob Proxy, but I ran into many difficulties trying to use it. As of this writing, Browsermob Proxy hasn't been updated in two years and has some strict limitations. For example, Browsermob proxy is unable to forward HTTPS requests which target an IP address directly instead of using a hostname. This is what my Android emulators do, regardless of the way my app is written.\n\nMitmproxy has an active ecosystem, modern tools, and lots of helpful documentation. I think we benefit from its wide use by researchers and security professionals rather than just testers.\n\nPerhaps mitmproxy's drawback is that it is written for Python developers, and hasn't been used much in testing to this date. I'd like to change this. The example code in today's edition will be in Javascript, and in a later edition I plan on introducing a package for interacting with mitmproxy from Java.\n\nAlright, on to the setup!\n\n### Setting up mitmproxy for iOS Simulators ###\n\nOur goal here is to run mitmproxy and have traffic from the iOS simulator pass through the proxy on it's way to the internet. Then we can inspect requests which mitmproxy logs. First we'll set up everything manually, and then put together the Python script which runs it all in the context of a test.\n\nWhenever in doubt, I suggest reading the first several pages of the [mitmproxy documentation](https://docs.mitmproxy.org/stable/concepts-howmitmproxyworks/). It helps to de-mystifty what we're trying to put together and may help you get back on the rails of this tutorial if you encounter issues.\n\nInstall mitmproxy. You may need to make sure you have Python3 and `pip3` installed first. Installing this way is required for the Javascript library we will be using later.\n```\nsudo pip3 install mitmproxy\n```\n\nAnd then run it:\n```\nmitmproxy\n```\nThe proxy runs on localhost port `8080` by default. The empty pane it displays should fill up with network traffic once we configure our simulator to point to it.\n\niOS simulators don't have their own settings for configuring a proxy, but they use the MacOS native network proxies. So the next step is to configure the entire host computer to use our mitmproxy. This means that the network traffic from whatever else our computer is doing will be mixed in with the simulator traffic, but we can filter it later.\n\nOpen the Network Preferences, click on `Advanced`, and click on the `Proxies` tab.\n\n

\n \"OSX\n

\n\nThere are several kinds of proxy available, we want to enable both the `Web Proxy (HTTP)` and `Secure Web Proxy (HTTPS)`. Check both proxy types and input the address of our mitmproxy server: `0.0.0.0` on port `8080`. Click `Ok` and `Apply`.\n\n

\n \"OSX\n

\n\nNow, all system traffic should be going through the proxy. Let's check real quick. Open [http://appium.io](http://appium.io), and the page should load. Look at the mitmproxy UI and we should see some related network requests (use arrow keys to scroll down).\n\nNow check [https://appiumpro.com](https://appiumpro.com), it should fail. This is an HTTPS website and we haven't set our system to trust the mitmproxy ssl certificate. We're actually not going to do that at all because we only need to set it up for the iOS simulator, and trusting the certificate on the OS level will leave us unsecured if we forget to untrust it after we finish testing.\n\nNext launch an iOS simulator and make the same check. Visiting [http://appium.io](http://appium.io) in mobile Safari should work, while [https://appiumpro.com](https://appiumpro.com) should fail to load.\n\nNow we want to install the mitmproxy ssl certificate on the simulator. Navigate to a special url hosted by mitmproxy: [http://mitm.it](http://mitm.it). Click on the Apple logo and follow the instructions to install the certificate on the simulator.\n\n__New instructions for iOS 12.2 and later:__\n*iOS 12.2 changed the UI for intalling certificates. The next step, if to open the Settings app and go to General->Profiles tap on the mitmproxy profile and tap 'install' in the upper right corner. Continue with the next step below.*\n\nAfter following prompts and tapping the `install` button in the upper right corner, there's still one more step! Often, the new certificate is not fully trusted by default. Open the Settings app and go to General->About->Certificate Trust Settings and enable full trust for the mitmproxy certificate. Now we should be able to view [https://appiumpro.com](https://appiumpro.com) in Safari.\n\nTraffic from Safari and native apps should now be logged in the mitmproxy UI!\n\n### Automating Certificate Installation ###\n\nI listed the manual steps above because it's easier to debug and figure out if your setup is correct.\nOnce running this all the time, if you have a new simulator to test on, you shouldn't have to perform this manual work. Appium can automate installing the certificate.\n\nUse the `mobile:installCertificate` command, passing it a base64 encoded string of the mitmproxy certificate. Appium will automate the settings app and trust it for you.\n\nOnce mitmproxy is run once, the certificate files are saved to your home directory, so that's where we read them from.\n\nHere's how I do it in our example test scripts:\n\n```javascript\nlet certificatePath = path.resolve(os.homedir(), '.mitmproxy', 'mitmproxy-ca-cert.pem')\nlet certificate = await fs.readFile(certificatePath)\nawait driver.executeScript('mobile:installCertificate', [{content: certificate.toString('base64')}])\n```\n\n### Capturing Traffic Within a Node.js Test Script ###\n\nLet's insert code to start mitmproxy before our test, then filter and access the captured traffic so we can make assertions on it.\n\nThere are a couple different approaches to this. What I really want is the ability to programmatically start mitmproxy and access the requests within my test. Mitmproxy is designed with a nicely-featured addon framework which supports plugins written in Python. The way the designers expects you to run mitmproxy is as a standalone service, and pass python script files to it using the `-s` commandline flag. We could write a python addon and start/stop the proxy as a separate shell process, but I want to avoid that if I can.\n\nInstead, I found a [great little Node.js module](https://github.com/jvilk/mitmproxy-node) which starts mitmproxy as a separate process and loads a special Python addon which opens a websocket connection so that we can get information in our Javascript code while the proxy is running.\n\nThe following test opens our usual demo app, [TheApp](https://github.com/cloudgrey-io/the-app/releases), opens our pickerwheel demo, and taps the button. This action results in the app sending a request to `history.muffinlabs.com` which returns some historical events for the app to display.\n\nThe test starts mitmproxy and logs all requests sent by the app to an array. At the end of the test, we verify that the request we expected the app to send was indeed sent.\n\n```javascript\nlet test = require('ava')\nlet fs = require('fs-extra')\nlet os = require('os')\nlet path = require('path')\nlet { remote } = require('webdriverio')\nlet Mitmproxy = require('mitmproxy').default\n\nlet proxy, driver\nlet interceptedMessages = []\n\n// this function will get called every time the proxy intercepts a request\nlet requestHandler = (message) => {\n let req = message.request\n console.log('************************************')\n console.log('mitmproxy intercepted a request')\n console.log(req.method)\n console.log(req.rawUrl)\n console.log(message.requestBody.toString())\n console.log('************************************')\n interceptedMessages.push(message)\n}\n\ntest.before(async t => {\n // start mitmproxy\n proxy = await Mitmproxy.Create(requestHandler, [], true, true)\n\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'iOS',\n platformVersion: '12.1',\n deviceName: 'iPhone XS',\n automationName: 'XCUITest',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip' // in order to download, you may need to install the mitmproxy certificate on your operating system first. Or download the app and replace this capability with the path to your app.\n },\n logLevel: 'silent'\n })\n\n // if the mitmproxy certificate was not installed manually already. Only needs to run once per simulator\n let certificatePath = path.resolve(os.homedir(), '.mitmproxy', 'mitmproxy-ca-cert.pem')\n let certificate = await fs.readFile(certificatePath)\n await driver.executeScript('mobile:installCertificate', [{content: certificate.toString('base64')}])\n})\n\ntest('getting the event for a day, via request to history.muffinlabs.com', async t => {\n let pickerDemo = await driver.$('~Picker Demo')\n await pickerDemo.click()\n let button = await driver.$('~learnMore')\n await button.click()\n // wait for alert\n let alertIsPresent = async () => {\n try { return await driver.getAlertText(); } catch { return false; }\n }\n await driver.waitUntil(alertIsPresent, 4000)\n await driver.dismissAlert()\n\n t.true(interceptedMessages.length > 0)\n t.true(interceptedMessages.some(m => m.request.rawUrl == 'https://history.muffinlabs.com/date/1/1'))\n})\n\ntest.after.always(async t => {\n t.log('shutting down')\n await proxy.shutdown()\n await driver.deleteSession()\n})\n```\n\n### Modifying Network Responses Recieved By The Device ###\n\nMitmproxy allows us to get even more clever. Just changing a couple lines in the test above, we can intercept the app's request to `history.muffinlabs.com` and rewrite the contents of the response to contain the data of our choosing.\n\nThis can be useful in getting the app into a predictable state. The current code chooses a random historical event to display every time, but we can respond with a single event of our choosing and force the app to display what we want. We can also force the UI to display specific values, so we can test long strings, negative numbers, error states, etc, without having to actually set up the necessary context on the backend. All this can be done from the test script!\n\n```javascript\nlet test = require('ava')\nlet { remote } = require('webdriverio')\nlet Mitmproxy = require('mitmproxy').default\n\nlet proxy, driver\nlet interceptedMessages = []\n\n// this is the response we will return from our proxy, instead of what the site usually returns\nlet injectedResponse = {\n date: \"January 1\",\n url: \"https://wikipedia.org/wiki/January_1\",\n data: {\n Events: [\n {\n year: \"2019\",\n text: \"Tests Passed\",\n }\n ]\n }\n}\n\nlet requestHandler = (message) => {\n message.setResponseBody(Buffer.from(JSON.stringify(injectedResponse), 'utf8'))\n}\n\ntest.before(async t => {\n proxy = await Mitmproxy.Create(requestHandler, [], true, true)\n\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'iOS',\n platformVersion: '12.1',\n deviceName: 'iPhone XS',\n automationName: 'XCUITest',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip' // in order to download, you may need to install the mitmproxy certificate on your operating system first. Or download the app and replace this capability with the path to your app.\n },\n logLevel: 'silent'\n })\n})\n\ntest(`insert our own event for a day, assert that it's displayed`, async t => {\n let pickerDemo = await driver.$('~Picker Demo')\n await pickerDemo.click()\n let button = await driver.$('~learnMore')\n await button.click()\n\n // wait for alert\n let alertIsPresent = async () => {\n try { return await driver.getAlertText(); } catch { return false; }\n }\n await driver.waitUntil(alertIsPresent, 4000)\n\n let alertText = await driver.getAlertText()\n await driver.dismissAlert()\n\n // assert that the alertText is the same as the packet we injected\n t.true(/Tests Passed/.test(alertText))\n})\n\ntest.after.always(async t => {\n t.log('shutting down')\n await proxy.shutdown()\n await driver.deleteSession()\n})\n```\n\nThat's all for this week. [Sample code is on github, as usual](https://github.com/cloudgrey-io/appiumpro/tree/master/javascript). Next week, I'll show how to create the same setup with Android Emulators (spoiler: it's harder).\n","metadata":{"lang":"all","platform":"iOS","subPlatform":"All Devices","title":"Capturing iOS Simulator Network Traffic with Appium","shortDesc":"Using a Proxy, we can capture the exact network packets sent from an iOS simulator and make assertions on their contents or timing. We can even modify the responses at will!","sponsor":"headspin","sponsorLink":"https://www.headspin.io/solutions/performance-optimization?utm_source=appiumpro&utm_medium=referral&utm_campaign=pilot","sponsorBlurb":"HeadSpin's packet engine does network capture on real devices and cell network radios, and delivers protocol analysis, PCAP, traceroute, and root cause analysis of common network issues that affect performance.","liveAt":"2019-03-27 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-ios-simulator-network-traffic-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/62-capturing-ios-simulator-network-traffic-with-appium","slug":"62-capturing-ios-simulator-network-traffic-with-appium","path":"/editions/62-capturing-ios-simulator-network-traffic-with-appium","news":null,"tags":["all","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0061.md","num":61,"rawMd":"\n\nWhen working on native apps which contain a webview, one needs to move the entire test session into the webview context in order to automate the elements within ([this process is explained in the Appium documentation](http://appium.io/docs/en/writing-running-appium/web/hybrid/)). In the case where _multiple_ webviews are present, how do we select the right one?\n\nGetting the list of all webviews using the `getContexts()` command will return a list which looks something like:\n\n```\n[\"NATIVE_APP\", \"WEBVIEW_41490.2\", \"WEBVIEW_41490.3\"]\n```\n\nEach of the webviews are named with a non-descriptive number, and the order is not guaranteed to be the same on every test run. If only there was a way to get more information about each webview when we call the `getContexts()` command.\n\nThis is where the `fullContextList` desired capability comes in. When set to `true`, the contexts returned by the `getContexts()` command are returned as a JSON string. Once parsed, each context is an object which includes not only the ID of the webview, but also the URL the webview is currently displaying and the Title of the page. Here's an example response when `fullContextList` is set to `true`:\n\n```\n[\n {\n \"id\":\"NATIVE_APP\"\n },\n {\n \"id\":\"WEBVIEW_41490.2\",\n \"title\":\"Appium Pro: The Awesome Appium Tips Newsletter\",\n \"url\":\"https://appiumpro.com/\"\n }\n]\n```\n\nIf multiple webviews are present in the current view, we can check the urls or titles of each so we can make sure to pick the right one.\n\nUnfortunately, this capability is only supported when testing on iOS devices. There is a way Appium could implement this functionality on Android, but the work has not been done yet. Volunteers are welcome!\n\nSelecting the right webview is a bit tricky in Java: you have to know that the contexts can be cast to `Map`, which is not obvious.\n\nHere's a sample test written in Java, and as usual the full file can be found in our [example code repository](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/).\n\n```java\n@Test\npublic void useFullContextList() throws MalformedURLException, InterruptedException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.1\");\n caps.setCapability(\"deviceName\", \"iPhone XS\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n caps.setCapability(\"app\", IOS_APP);\n caps.setCapability(\"fullContextList\", true);\n\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n // get to webview screen\n wait.until(ExpectedConditions.presenceOfElementLocated(hybridScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(urlInput)).sendKeys(\"https://appiumpro.com\");\n driver.findElement(goButton).click();\n\n Optional> appiumProWebview;\n do {\n Thread.sleep(100);\n // get full list of contexts\n Set> contexts = driver.getContextHandles();\n appiumProWebview = contexts.stream()\n .filter(c -> !c.get(\"id\").equals(\"NATIVE_APP\"))\n .filter(c -> c.get(\"url\").equals(\"https://appiumpro.com/\"))\n .findFirst();\n } while (!appiumProWebview.isPresent());\n\n\n driver.context(appiumProWebview.get().get(\"id\").toString());\n\n // now we know we're in the webview context which was pointing to \"appiumpro.com\"\n assertTrue(driver.getTitle().equals(\"Appium Pro: The Awesome Appium Tips Newsletter\"));\n}\n```\n","metadata":{"lang":"all","platform":"iOS","subPlatform":"All Devices","title":"How to Accurately Select Webviews Using the `fullContextList` Capability","shortDesc":"The `fullContectList` desired capability can be used to select a specific webview context displayed on a device, if more than one are present.","liveAt":"2019-03-20 10:00","canonicalRef":"https://www.headspin.io/blog/how-to-accurately-select-webviews-using-the-fullcontextlist-capability"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/61-how-to-accurately-select-webviews-using-the-fullcontextlist-capability","slug":"61-how-to-accurately-select-webviews-using-the-fullcontextlist-capability","path":"/editions/61-how-to-accurately-select-webviews-using-the-fullcontextlist-capability","news":null,"tags":["all","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0060.md","num":60,"rawMd":"\n\nReader Venkata recently asked us to compare and contrast the different selector strategies available. If you have the luxury to choose, which is the best pick?\n\nWe've written about element selectors quite a bit already. For those new to the subject, we have a good overview of [what selector strategies are and how to use them](https://appiumpro.com/editions/20).\nI'm going to omit mention of Web and Hybrid tests here, and focus just on the selector strategies provided by Appium for native iOS and Android testing using the UiAutomator2 and XCUITest drivers.\n\nWithout further ado, here's our prioritized list of locator strategies:\n\n1. `accessibility id`\n2. `id`\n3. `xpath`\n4. `class name`\n5. Locators interpreted by the underlying automation frameworks, such as: `-android uiautomator`, `-ios predicate string`, `-ios class chain`\n6. `-image`\n\nNow don't go running to your fellow testers using this list to settle an argument. Every app is different, and assembling this list is like making a list of the \"best\" pieces in chess: they all have their uses.\n\n### 1. `accessibility id` ###\n\nThat this is the top choice should surprise nobody. If you have the option of using accessibility IDs, use them. Normally an app developer needs to add these specifically to UI elements in the code. The major benefit of accessibility IDs over just the `id` locator strategy is that while app developers add these IDs for testing, users with handicaps or accessibility issues benefit. People who use screen readers or other devices, and algorithms which inspect UIs, can better navigate your app!\n\nOn Android, this locator strategy uses the accessibility `contentDescription` property.\n\nOn iOS, this locator strategy uses the accessibility identifier. Here's something surprising: in the XCUITest driver, the `accessibility id`, `id`, and `name` locator strategies are all identical. They are implemented the same way. Go ahead and try switching your locator strategies, you will get the same results. This may change in the future, but for now you can find an element using the name or text in it because iOS has many ways in which it sets a default accessibility identifier if one is not provided by the developer.\n\n### 2. `id` ###\n\nElement IDs need to be added by a developer, but they allow us to pinpoint the exact element in the app we care about, even if the UI changes appearance. The drawback is you need to be able to talk to your developers. Many testing teams do not have this luxury.\n\nThis locator strategy is pretty similar to `accessibility id` except that you don't get the added benefit of accessibility.\n\nAs noted above, on iOS, this is actually identical to `accessibility id`.\n\nOn Android, the `id` locator strategy is implemented using Resource IDs. These are usually added to UI elements manually by app developers, though are not required, so many app developers will omit them if they don't think they're important.\n\n### 3. `xpath` ###\n\nNow this is contentious. XPath is the most expressive and commonly accepted locator strategy, but Appium developers have long warned of its drawbacks. [This earlier edition of Appium Pro](https://appiumpro.com/editions/8) goes to some depth in explaining why XPath should be avoided.\n\nDespite Appium developers warning against XPath's low performance for years, it still seems to be the most popularly used locator strategy. This is probably because there are many selections that can't easily be made any other way. For example, there's no way to select the parent of an element using the simple `id` selectors. The benefit of being able to express more complicated queries must outweigh the cost to performance for all but the testers whose apps have such large XML element hierarchies that XPath is completely unusable. Those who need performance _and_ flexibility can use the native locator strategies in the next section (and I'll explain why I put them at #4 instead of #3).\n\nXPath selectors can be very brittle, but they can be responsibly wielded to great effect if you follow the guidelines of [this earlier Appium Pro edition](https://appiumpro.com/editions/35). Being intentional and carefully picking selectors rather than taking whatever an inspector provides can mitigate the brittleness.\n\nI think part of the popularity of XPath stems from its use with Selenium and web development, as well as it being the default of many tutorials and inspection tools. When working on Appium I always expected our XPath handling to break more often, but I remember few bugs, probably the benefit open source XPath libraries built for more generalized use.\n\nThe Android OS provides a useful `dumpWindowHierarchy` which gives us an XML document of all the elements on the screen. From there we apply the XPath query and find elements.\n\niOS does not supply a method of getting the entire hierarchy. Appium's implementation starts at the root application element and recurses through each element's children and populates and XML document which we can then apply the XPath query to.\n\nI still think XPath is unintuitive, especially for those new to programming, but at least it's a well-accepted industry standard.\n\n### 4. `-android uiautomator`, `-ios predicate string` or `-ios class chain` ###\n\nI call these the \"native\" locator strategies because they are provided by Appium as a means of creating selectors in the native automation frameworks supported by the device. These locator strategies have many fans, who love the fine-grained expression and great performance (equally or just slightly less performant than `accessibility id` or `id`).\n\nThese locator strategies are crucial for those who have UIs which escape the grasp of the other locator strategies or have an element tree which is too large to allow the use of XPath. In my view, they have several drawbacks.\n\nThese native locator strategies require a more detailed understanding of the underlying automation frameworks. Uiautomator and XCUITest can be hard to use, especially for those less familiar with Android and iOS specifics. These locator strategies are not cross platform, and knowing the ins-and-outs of both iOS and Android is challenging.\n\nIn addition, the selectors passed to these native locator strategies are not directly evaluated by the mobile OS. Java, Kotlin, Objective C and Swift all lack an `eval` function which would allow interpreting a string of text as code. When you send an `android uiautomator` selector to Appium, the text passes through a very simplistic parser and uses [Reflection](https://docs.oracle.com/javase/8/docs/technotes/guides/reflection/index.html) to reconstruct the objects referenced in the text. Because of this, small mistakes in syntax can throw off the entire selector and only the most common methods are supported. This system is unreliable and often encounters difficult bugs.\n\nIf Android or iOS change the testing classes in new updates, your selectors might need to be updated. Using XPath, Appium will keep track of the OS updates and your selectors should keep working.\n\nA personal quibble I have with these locator strategies is that you are essentially writing a different programming language (such as Java) inside of a string in your test code. Your text editor will not offer syntax highlighting or semantic analysis inside of these queries which makes them harder to maintain.\n\nOn the other hand, sometimes there's just no other way, and for those who are proficient in these methods XPath can seem clumsy in comparison.\n\n### 5. `-image` ###\n\nThis locator strategy is pretty nifty. Supported on all platforms and drivers, you can pass an image file to Appium and it will try to locate the matching elements on the screen. We already went over this in [Appium Pro edition 32](https://appiumpro.com/editions/32).\n\nThis supports fuzzy matching, and certainly has its applications, but you should only use it when the other locator strategies aren't working. It may not always behave deterministically, which mean it can be a source of test flakiness. Also, unless you have a special code editor, they may be a little more difficult to work with and maintain.\nOf course, visual UI changes will always break `-image` selectors, whereas the other selectors only break when the element hierarchy changes.\n\nThat's it! Feel free to drop us a line if you feel passionately about one of these strategies!\n","metadata":{"lang":"all","platform":"All Platforms","subPlatform":"All Devices","title":"How to Pick the Right Locator Strategy","shortDesc":"A look at the pros and cons of the main locator strategies supported on iOS and Android by Appium, and a loose ranking of which should be defaulted to if given a choice.","liveAt":"2019-03-13 10:00","canonicalRef":"https://www.headspin.io/blog/how-to-pick-the-right-locator-strategy"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/60-how-to-pick-the-right-locator-strategy","slug":"60-how-to-pick-the-right-locator-strategy","path":"/editions/60-how-to-pick-the-right-locator-strategy","news":null,"tags":["all","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0059.md","num":59,"rawMd":"\n\nOne of the mobile-specific UI controls that we tend to either love or hate is the \"picker wheel\"--the mobile version of the \"select box\". iOS and Android by default do things a little differently, and this article addresses strategies for automating the iOS-specific picker wheel.\n\nTo work with a picker wheel, I've added a few to [The App](https://github.com/cloudgrey-io/the-app). Basically, there's now a new view that asks us to select a month and a day, and then, after tapping a button, tells us a random historical event that happened on that date in some previous year. (I'm using a freely available API I found at [history.muffinlabs.com](https://history.muffinlabs.com/)--thanks, Colin!) The new feature looks like this:\n\n

\n \"The\n

\n\nWe've actually got two picker wheels; one to select the month, and one to select the day. So, how do we automate this?\n\n### Method 1: SendKeys\n\nThe most straightforward way is just to use `driver.sendKeys`. What, `sendKeys`? How does that make sense? Selecting a picker wheel value uses a finger gesture in real life, not any entering of text! All of that is true. However, the main purpose of `sendKeys` is to allow the user to input data into a form control, and that is exactly what we're doing here. In fact, Selenium uses `sendKeys` to set the value of an HTML `select` field, which is more or less what we're doing here.\n\nOK, so now that we've established it's not totally crazy, how does it work? Basically, just use the displayed text value of the picker wheel item you want to select, and that's it. For example, if we have our picker wheel element already found, we could do this:\n\n```java\npickerWheelElement.sendKeys(\"March\");\n```\n\nIn the example of our test app, this would set the value of the picker wheel to \"March\".\n\nIf at all possible, this is the preferred method, since it is simple and fast. However, there might be times where you don't _know_ the value of the picker wheel item beforehand; in this case, check out the alternative method below.\n\n### Method 2: selectPickerWheelValue\n\nAppium also makes available a `mobile:` method called `selectPickerWheelValue`, which is primarily useful for navigating a picker wheel using forward-and-back gestures. The command takes 3 parameters:\n\n* `order`: whether you want to go forward (\"next\") or backward (\"previous\") in the list of options.\n* `offset`: how far (in ratios of the picker wheel height) you want the click to happen. The default is `0.2`.\n* `element`: the internal ID of the picker wheel element we're working with.\n\nOnce we know what we want to do, we simply have to construct our typical `executeScript` call with these parameters. For example, if we have the same element as we just saw above:\n\n```java\nHashMap params = new HashMap<>();\nparams.put(\"order\", \"next\");\nparams.put(\"offset\", 0.15);\nparams.put(\"element\", ((RemoteWebElement) pickerWheelElement).getId());\ndriver.executeScript(\"mobile: selectPickerWheelValue\", params);\n```\n\nThis will move the value of the picker wheel to the next one in the list!\n\n### Full Example\n\nThat's all there is to having full automation control over iOS's picker wheels. You can check out the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition059_Picker_Wheel.java) I produced to demonstrate these concepts, and it's also reproduced here, with comments, for your perusal. Enjoy!\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport java.util.List;\nimport org.hamcrest.Matchers;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.Alert;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.remote.RemoteWebElement;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition059_Picker_Wheel {\n\n private String APP_IOS = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.9.0/TheApp-v1.9.0.app.zip\";\n\n private IOSDriver driver;\n private WebDriverWait wait;\n\n private static By pickerScreen = MobileBy.AccessibilityId(\"Picker Demo\");\n private static By pickers = MobileBy.className(\"XCUIElementTypePickerWheel\");\n private static By learnMoreBtn = MobileBy.AccessibilityId(\"learnMore\");\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"app\", APP_IOS);\n\n driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testPicker() {\n // get to the picker view\n wait.until(ExpectedConditions.presenceOfElementLocated(pickerScreen)).click();\n\n // find the picker elements\n List pickerEls = wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(pickers));\n\n // use the sendKeys method to set the picker wheel values directly\n pickerEls.get(0).sendKeys(\"March\");\n pickerEls.get(1).sendKeys(\"6\");\n\n // trigger the API call to get date info\n driver.findElement(learnMoreBtn).click();\n\n // verify info was retrieved for the correct date\n wait.until(ExpectedConditions.alertIsPresent());\n Alert alert = driver.switchTo().alert();\n Assert.assertThat(alert.getText(), Matchers.containsString(\"On this day (3/6) in\"));\n\n // clear the alert\n alert.accept();\n wait.until(ExpectedConditions.not(ExpectedConditions.alertIsPresent()));\n\n // use the selectPickerWheelValue method to move to the next value in the 'month' wheel\n HashMap params = new HashMap<>();\n params.put(\"order\", \"next\");\n params.put(\"offset\", 0.15);\n params.put(\"element\", ((RemoteWebElement) pickerEls.get(0)).getId());\n driver.executeScript(\"mobile: selectPickerWheelValue\", params);\n\n // and move to the previous value in the 'day' wheel\n params.put(\"order\", \"previous\");\n params.put(\"element\", ((RemoteWebElement) pickerEls.get(1)).getId());\n driver.executeScript(\"mobile: selectPickerWheelValue\", params);\n\n // trigger the API call to get date info\n driver.findElement(learnMoreBtn).click();\n\n // and finally verify info was retrieved for the correct date (4/5)\n wait.until(ExpectedConditions.alertIsPresent());\n alert = driver.switchTo().alert();\n Assert.assertThat(alert.getText(), Matchers.containsString(\"On this day (4/5) in\"));\n alert.accept();\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"How to Automate Picker Wheel Controls","shortDesc":"Sometimes working with iOS's pesky picker wheels can be an automation challenge. There are two methods for dealing with picker wheels, including a special method custom-designed to work with picker wheels even when the desired value is not known beforehand.","liveAt":"2019-03-06 10:00","canonicalRef":"https://www.headspin.io/blog/how-to-automate-picker-wheel-controls"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/59-how-to-automate-picker-wheel-controls","slug":"59-how-to-automate-picker-wheel-controls","path":"/editions/59-how-to-automate-picker-wheel-controls","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0058.md","num":58,"rawMd":"\n\nAppium has the ability to start Android emulators and iOS simulators in a \"headless\" mode. This means that the devices won't have any graphical user interface; you won't see them on your desktop, but they will still be running silently, testing your app.\n\nActually, the devices won't run silently. They will still emit sounds through your speakers, access the microphone, or anything else they usually do, you just won't have a window on your screen which shows what they're doing, and you won't get those interactive buttons.\n\nMost surprisingly, the emulators and simulators still render the app UI, they just don't display it. This means that the video recording and screenshot commands still work as usual. The only thing we lose when running in headless mode is the ability for a user to manually intervene and manipulate a device during a test session.\n\nThis style of headless virtual device is a little different from headless browsers, which do not render the UI and therefore enable decreased test times because the costly process of deciding how to lay everything out on the screen is skipped. Sadly, running headless emulators and simulators will save a little bit of system resources by not running the actual windowed application, but compared to the CPU and memory required to run the device, this is minimal.\n\nHeadless simulators and emulators should run exactly the same as their graphical counterparts, but I encountered an issue running our usual [example app](https://github.com/cloudgrey-io/the-app) in a headless Android emulator! The issue seems to have something to do with ReactNativeNavigation, but this shows that _something_ is different about Android emulators in headless mode.\n\nHeadless emulators and simulators are most useful for device farms which run devices on servers which have no graphical window manager in the first place. Though rare, [running OSX fully headless is possible](http://osxdaily.com/2018/09/02/access-login-console-terminal-mac/), in which case it is convenient that simulators can still be run.\n\nDesktop tools can also take advantage of this features. Imagine a tool like Appium Desktop which starts its own emulators and simulators but doesn't want to confuse the user by popping up extra windows. The tool could use the headless feature to hide these windows from the user.\n\n(In terms of using headless devices on my personal computer, the best use case I discovered is to stop the iOS simulator from stealing my window focus and popping up over whatever I'm doing whenever a test starts up.)\n\n## How to Run Headless Emulators and Simulators\n\nIt's simple! Just set the following desired capability to `true`:\n\n```java\ncaps.setCapability(\"isHeadless\", true);\n```\n\nIf you're testing on android, you will have to provide the `avd` desired capability so that Appium starts the emulator with the necessary arguments. Not specifying the `avd` results in Appium using the current emulator already running, and of course if it's already running with a UI, the UI will stay visible.\n\niOS has no such issue; if you set `isHeadless` to `true`, then Appium will shut down any currently running simulators and restart them in headless mode.\n\nUnless you instructed Appium to shut down the device afterwards, the headless emulators and simulators will still be running after your session, but you won't see them unless you check via the `simctl` utility or `adb`.\n\n## How Does it Work? ##\n\nFor iOS simulators, the actual simulator is a separate process from the `Simulator.app` app. When Appium starts an iOS simulator normally, it has to start the simulator using the `simctl` utility, and then it launches `Simulator.app`. When running in headless mode, Appium just skips the second step. If you want to start playing with a simulator which is running in headless mode, launch `Simulator.app` and a window will be displayed for all currently running simulators. Warning: quiting `Simulator.app` shuts down all simulators!\n\nFor headless Android emulators, Appium starts the emulator via ADB as usual, and adds the `-no-window` flag. Documentation can be found [here](https://developer.android.com/studio/run/emulator-commandline).\n","metadata":{"lang":"all","platform":"All Platforms","subPlatform":"All Devices","title":"How to Test on Headless Emulators and Simulators with Appium","shortDesc":"Using the 'isHeadless' desired capability, you can tell Appium to run emulators and simulators in the background without their graphical user interfaces.","liveAt":"2019-02-27 10:00","canonicalRef":"https://www.headspin.io/blog/how-to-test-on-headless-emulators-and-simulators-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/58-how-to-test-on-headless-emulators-and-simulators-with-appium","slug":"58-how-to-test-on-headless-emulators-and-simulators-with-appium","path":"/editions/58-how-to-test-on-headless-emulators-and-simulators-with-appium","news":null,"tags":["all","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0057.md","num":57,"rawMd":"\n\nOne of the prime problems facing many automation engineers is how to start testing a new app. Often you're just given a build of the app and a set of user flows to create automated test scenarios for--no specific instructions from the developers on how individual elements might be accessed from automation. The question becomes, how do you turn UI components on the screen into Appium's [locator strategies](https://appiumpro.com/editions/8)? I'm sure many of you have used the wonderful [Appium Desktop](https://github.com/appium/appium-desktop) as a tool for inspecting your native mobile applications. Appium Desktop's Inspector has a nice point-and-click interface you can use to get information about on-screen elements, including suggested locator strategies and selectors. More on that, and other tools for inspecting native apps, in a future edition!\n\nFor now, outside the world of native apps, the picture gets more complicated. What if you have been asked to automate a web app using mobile Safari or Chrome? Or what if your app is a hybrid app? How do you know how to find the web elements inside the app's webview? If you're familiar with Selenium, you might have started looking for some \"developer tools\" option in Safari or Chrome on your mobile device, only to be left empty-handed. The trick is to mix in the desktop versions of those browsers on your computer!\n\n## Inspecting Mobile Safari and Hybrid Apps on iOS\n\nIf you're running an iOS simulator with Safari open, and have navigated to the web page you want to automate, inspecting is as easy as opening up the desktop version of Safari, and going to the \"Develop\" menu in the menu bar:\n\n

\n \"Safari's\n

\n\n(Don't see the Develop menu? Go to Safari's preferences, then the \"Advanced\" tab, and make sure \"Show Develop menu in menu bar\" is checked)\n\nFrom that menu, you should see a new entry in the list of devices, reflecting the open simulator. Navigating into that menu and clicking on the appropriate website will pop up an inspector window for you:\n\n

\n \"Inspecting\n

\n\nYou can now do everything you would normally do in Safari's developer tools, including exploring the HTML hierarchy (complete with colored rectangles being drawn over the corresponding elements in the simulator)! You will, of course, need to know how to turn this information into Appium locators / selectors. For example, in the image above, I've selected the menu element for the mobile version of the Appium Pro website. It happens to have a unique class, so I could find this element with a simple `driver.findElement(By.cssSelector(\".toggleMenu\"))`.\n\n(If you're brand new to web development, it's worth getting familiar with reading HTML, since that is how you'll know which locator strategies can be used to find an element, and the particular selector that will do the trick).\n\nIf you're on a real device, you'll need to open up the Advanced settings menu under the Safari preferences, and make sure that \"Web Inspector\" is turned on. Otherwise, you won't see your device in the Develop menu.\n\nThe good news is that if you're running a hybrid app, this same strategy should work without any additional setup--you'll simply see your webview as one of the inspectable pages in the Develop menu!\n\n## Inspecting Mobile Chrome\n\nWe can employ a similar strategy for Chrome, though in my opinion the experience is not quite as seamless. Again, the first thing you'll do is load up your page in the Chrome browser on your emulator or device. Then, open up the desktop version of Chrome, and navigate to [chrome://inspect#devices](chrome://inspect#devices). You should see a window that looks like this:\n\n

\n \"Chrome's\n

\n\nFrom here, you should see your emulator or device listed, and if you're lucky, the specific page you're looking at. Now you have simply to click \"inspect\" and you'll be on your way.\n\nIf you don't see your device or page, it could be for a few different reasons. First of all, emulators need to forward the remote debugging port to the host system so that desktop Chrome (which is running on the host system) can access that port in order to facilitate the inspection. So the first thing to try is the ADB command which forwards the appropriate port:\n\n```\nadb forward tcp:9222 localabstract:chrome_devtools_remote\n```\n\nRunning this command should get your device to pop up in the Inspect list. Now, once you've clicked \"inspect\", you'll again get a full-featured inspector window with all the developer tools your little robot heart fancies:\n\n

\n \"Inspecting\n

\n\nThe only downside to the Chrome inspector is that hovering over elements in the HTML source does not highlight the corresponding element on the device screen. But anyway, you can use it in the same way I described above for Safari to determine your locator strategies and selectors.\n\nFor hybrid apps, you might not need to do anything else. If you have opened up your app, navigated to the view containing the webview, and see the webview contents in the Inspect window, then great! If not, it could be that the webview has not turned on debugging. For security, Android requires that each webview explicitly allows debugging, otherwise debugging via Chrome will not work.\n\nAs a convenient way of applying this setting to all webviews in your app, you (or your developer) can simply ensure that the following method is called on the `WebView` class:\n\n```java\nWebView.setWebContentsDebuggingEnabled(true);\n```\n\nRe-compile your app, launch it, and you should now see your webview in a debuggable, inspectable state!\n","metadata":{"lang":"all","platform":"All Platforms","subPlatform":"All Devices","title":"How to Determine Element Locators For Mobile Web and Hybrid Apps","shortDesc":"When we're automating mobile web and hybrid apps, it can be hard to know which selectors and locator strategies to use to find particular elements. Tools like Appium Desktop or uiautomatorviewer are no help here, but that's for a good reason: our browsers come with all the tools we need to solve this problem already.","liveAt":"2019-02-20 10:00"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/57-how-to-determine-element-locators-for-mobile-web-and-hybrid-apps","slug":"57-how-to-determine-element-locators-for-mobile-web-and-hybrid-apps","path":"/editions/57-how-to-determine-element-locators-for-mobile-web-and-hybrid-apps","news":null,"tags":["all","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0056.md","num":56,"rawMd":"\n\n## The Activity ##\n\nThe Activity is a core concept of Android apps and fundamentally shapes app architecture and UI design. Therefore, our testing strategy can benefit from understanding and incorporating activities.\n\nAn Android app is not a single experience with a set beginning and end, it is a collection of activities which can be entered and left at will. UI designers think of apps in terms of \"screens\", which is a pretty close analog to activities. An activity provides the developer with a window in which to draw the UI, and each screen of an app typically maps to a single activity. As a user moves from one screen to another, the app launches the next activity from the current one.\n\n

\n \"Each\n

\n\nEach of the above screens from the Android settings app, including the main menu, is a separate activity.\n\nThe movement between activities isn't limited to staying within a single app. A single user session on a phone could start with a social networking app, which opens a web browser, which opens up an email app, which might open the camera app. An app may launch an activity from any other app, passing information along in the process. While an app UI may direct a user through activities in an established flow, nothing prevents activities from being launched outside this flow.\n\nThe reason Android is designed this way is so that apps can send information to each other, or supply actions which can be reused. Launching the camera capture activity from your messenger app is much easier and better for the developer ecosystem than requiring each app developer to write their own camera activity. Desktop apps have some semblance of this functionality when it comes to those pesky `mailto:` links and the default file-picker.\n\nAll activities in an Android app must be declared in the app's `AndroidManifest.xml` file. Apps launch their own activities the same as they launch activities in other apps, using the `startActivity()` method.\n\nHere's an example of two activities being defined for the Android default camera app in its manifest file:\n```xml\n\n \n \n \n \n \n \n \n \n \n\n\n\n \n \n \n \n\n```\n\nAlmost every Android device has a dedicated \"back\" button. It's kind of strange, no? We are used to web browsers having back buttons, but how did every app developer agree to use the same built-in back button and have it perform the same way? They didn't need to, every time a new activity is launched, the device records the previously running activity, which will be relaunched when the back button is pressed. This and other familiar Android design features stem from the concept of the Activity.\n\n## Intents ##\n\nActivities are launched via Intents. There are two types of intents: explicit and implicit. An explicit intent uses the name of the app along with the name of the activity (as declared in the app's manifest file) to launch the intended activity. Explicit intents are used by developers for moving between activities within a single app. You wouldn't want to start an activity in another developer's app using the activity name, because the developer might change their app's internal structure and they didn't build those activities with your interests in mind.\n\nActivities can be launched on connected devices from the commandline using ADB and the `am` command (short for Activity Manager). The following is an example of launching the default camera app using an explicit intent. We specify the package name of the camera app and the name of the activity, from the manifest file excerpt above.\n\n```bash\nadb shell am start -n com.android.camera2/com.android.camera.CaptureActivity\n```\n\n

\n \"Launch\n

\n\nWhen developers build their activities to be used by other apps, they label their activities so they may be targeted by implicit intents. As the name implies, an implicit intent doesn't specify a particular app or activity to launch, instead it specifies the __sort__ of thing the user would like to do. Activities can be given an `action` and `category`. Examples of common android intents are :`android.intent.action.SEND`, `android.intent.action.VIEW`, and `android.intent.action.EDIT`. These actions describe the sorts of things a user might want to do. When an app launches an activity using an implicit intent the Android operating system looks through all the activities of all the apps on the device (using their manifest files) and finds the activities which declared that they respond to the action (and category) specified in the intent. If a single activity is found which handles the given action, the activity is launched and the resulting UI shown to the user.\n\nWe can launch the camera app using implicit intents this time, specifying the IMAGE_CAPTURE action:\n\n```bash\nadb shell am start -a android.media.action.IMAGE_CAPTURE\n```\n\nWe can also launch the camera app directly to video capture, rather than still.\n\n```bash\nadb shell am start -a android.media.action.VIDEO_CAMERA\n```\n\n

\n \"Launch\n

\n\nIf multiple apps support the same activity, the user is given a choice as to which activity they'd like to handle the intent. This decision is probably familiar to all Android users:\n\n

\n \"Multiple\n

\n\nPictured above, my emulator has multiple apps which respond to a user's attempt to set a home screen wallpaper, which I triggered using the following ADB command:\n\n```bash\nadb shell am start -a android.intent.action.SET_WALLPAPER\n```\n\nInformation is also passed along with an intent. A `data` URI can be specified, for example: to open a specific webpage or photo on the device. The form of the URI determines which activities may respond to the intent, just like with actions. This is how \"deep linking\" is implemented, the \"deep links\" are just intents which are matched to activities.\n\nThis example opens the default web browser to the url provided, since on my emulator the Chrome browser is the only app that responds to the `android.intent.action.VIEW` action and also accepts URIs starting with `http`:\n\n```bash\nadb shell am start -a android.intent.action.VIEW -d http://appiumpro.com\n```\n\n

\n \"Launch\n

\n\nThe following URI doesn't start with `http` like a web url does, but Google Maps will recognize it and open the given coordinates:\n\n```bash\nadb shell am start -a android.intent.action.VIEW -d \"geo:46.457398,-119.407305\"\n```\n\n

\n \"Launch\n

\n\nIn addition to the provided URI, special \"extra\" parameters can be added to intents. These are rather particular to each activity and are outlined in the [documentation](https://developer.android.com/guide/components/intents-filters). Here's a quick example which uses the `android.intent.action.SEND`, otherwise known as \"Share\", and sends some optional text which we'd like to share.\n\n```bash\nadb shell am start -a \"android.intent.action.SEND\" -e \"android.intent.extra.TEXT\" \"AppiumPro\" -t \"text/plain\"\n```\n\n

\n \"Sent\n

\n\n## For Appium Testers ##\n\nGreat, we know all about Android activities and intents, and have seen how to launch activities and send intents using ADB, but how do we fit this into our mobile test automation tool belt, and how does Appium interact with activities?\n\nYou may have seen the following in Appium server logs before:\n\n```\n[debug] [AndroidDriver] Parsing package and activity from app manifest\n[ADB] Using apkanalyzer from /Users/jonahss/Library/Android/sdk/tools/bin/apkanalyzer\n[debug] [ADB] Starting '/Users/jonahss/Library/Android/sdk/tools/bin/apkanalyzer' with args [\"-h\",\"manifest\",\"print\",\"/var/folders/mj/pgmgyh_95rsbs10yl46b73zc0000gn/T/2019112-4784-1duc7h5.tlh8/appium-app.apk\"]\n[ADB] Package name: 'io.cloudgrey.the_app'\n[ADB] Main activity name: 'io.cloudgrey.the_app.MainActivity'\n[debug] [AndroidDriver] Parsed package and activity are: io.cloudgrey.the_app/io.cloudgrey.the_app.MainActivity\n```\n\nBy default, Appium parses the AndroidManifest.xml file in the `.apk` file to find the main activity and package name. Appium then launches the app under test using these parameters.\n\nIf you'd like to launch a different activity in your app, you can use the `appPackage` and `appActivity` desired capabilities. Intents can also be described using the following desired capabilities: `intentAction`, `intentCategory`, `optionalIntentArguments`, `intentFlags`.\n\nThe `optionalIntentArguments` are the \"extra\" parameters which can be sent with intents, outlined in the last ADB example about sharing text.\n\n`intentFlags` are a complex set of extra commands which can alter the way activities are launched. They mostly have to do with how the newly launched activity relates to a task. Tasks are another Android concept, where each task is a collection of Activities. Intent flags are pretty technical and beyond the scope of this article, but you can learn more about them from the [Android documentation](https://developer.android.com/guide/components/activities/tasks-and-back-stack).\n\nActivities can also be launched mid-test, rather than at the beginning of a session. This can be done using the [`startActivity` command](http://appium.io/docs/en/commands/device/activity/start-activity/):\n\n```java\ndriver.startActivity(new Activity(\"com.example\", \"ActivityName\"));\n```\n\nA common requirement for mobile automation is having the app under test hit a development API server. Initially, a small team can publish a development build of the app which points to a development API. When the team grows, you may find yourself with multiple environments to target. Rather than building a different app for each environment, parameters such as the API url can be set via extra intent arguments. This way, an Appium test can specify app settings using the `optionalIntentArguments` desired capability, and no change needs to be made to the UI of the app.\n\nMost UI tests tend to test the same screen over and over in different ways, trying different form inputs, etc. Sometimes the screen we are testing is deep in the app, requiring many steps such as search and navigation to get to. Rather than run through the first screens of your app over and over, you can use intents to navigate directly to the activity you'd like to test. This can speed up overall test suite execution times.\n\nNot sure which activities and intents your app supports? All activities must be declared in the app's `AndroidManifest.xml` file. If you have your `.apk` file, you can find the manifest by opening any android project in Android Studio and choosing `Build >> Analyze APK ...` from the top navigation bar. Choose your APK file from the file system and then view the `AndroidManifest.xml` file. All activities and the intents they respond to will be listed in `` nodes.\n\n## Final Remarks ##\n\nThe Android documentation for Activities and Intents are well written and pretty easy to understand. I glossed over a lot of app development specifics to get to the content which is relevant to Appium automation. Read these for more detail:\n - [App Manifest Overview](https://developer.android.com/guide/topics/manifest/manifest-intro)\n - [Introduction to Activities](https://developer.android.com/guide/components/activities/intro-activities)\n - [Intents and Intent Filters](https://developer.android.com/guide/components/intents-filters)\n\nIt has come to my attention that these days, the concept of a \"single activity app\" is gaining popularity among Android developers. Frameworks are being made which build an entire app within a single Activity. This may limit your ability to jump into specific activities in order to speed up test suites.\n","metadata":{"lang":"java","platform":"Android","subPlatform":"All Devices","title":"What Appium Users Need to Know about Android Activities and Intents","shortDesc":"What are Activities and Intents? How they shape Android app design and how we can improve our test suites by being aware of them.","liveAt":"2019-02-13 10:00","canonicalRef":"https://www.headspin.io/blog/what-appium-users-need-to-know-about-android-activities-and-intents"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/56-what-appium-users-need-to-know-about-android-activities-and-intents","slug":"56-what-appium-users-need-to-know-about-android-activities-and-intents","path":"/editions/56-what-appium-users-need-to-know-about-android-activities-and-intents","news":null,"tags":["java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0055.md","num":55,"rawMd":"\n\nI recently wrote about [how to access Android Logcat logs](https://appiumpro.com/editions/53) using the `getLogs()` command, but Appium also supports subscribing to device logs as a stream of events via a WebSocket connection. Rather than getting a chunk of recent log lines every time a synchronous call to `getLogs()` is made, you can assign a function to be called every time a new log line is generated. Overall, this results in a better programming model for most cases, and only adds a little more code depending on the language.\n\nAppium needs to be told to start its WebSocket server before we can begin listening for log messages. This is done using one of the [special `mobile:` commands](http://appium.io/docs/en/commands/mobile-command/).\n\n```java\ndriver.executeScript(\"mobile:startLogsBroadcast\");\n```\n\nAppium will start a WebSocket server and accept connections on the same host and port to which you connected to start the test session. The path is different though; instead of `/wd/hub` the path starts with `/ws`:\n - `/ws/session/{sessionId}/appium/device/logcat` for Android logcat logs\n - `/ws/session/{sessionId}/appium/device/syslog` for iOS device logs\n\nWebSocket URLs start with `ws://` instead of `http://`. Running locally with the default Appium port, the WebSocket URLs would be:\n - `ws://localhost:4723/ws/session/{sessionId}/appium/device/logcat` on Android\n - `ws://localhost:4723/ws/session/{sessionId}/appium/device/syslog` on iOS\n\nUsing a WebSocket client and connecting to these URLs, we can assign a function to perform whatever action we want whenever a new log message is received.\n\nThere are several advantages of streaming logs this way. By using streams, we are only holding a single log message in memory at a time. If the device logs are very long, calling `getLogs()` and loading all the device logs could take a while and even consume too much memory on our test servers. We are also able to make sure that we have the latest logs available. If an exception is thrown in our test code, or a crash occurs in the app, device, or Appium server, we can be sure that we have the last log message which made its way through our system (which is most likely to contain information about the crash!). This will work better than catching exceptions and then calling `getLogs()`, for by then it may be too late to reach the device. We can also abstract the log-handling code further from our test code. By writing multiple WebSocket clients, we can change how logs are handled depending on the system environment.\n\nFor our Java example, I used [TooTallNates's java-websocket package](https://github.com/TooTallNate/Java-WebSocket).\nI'm also including a Javascript example using [websocket-stream](https://www.npmjs.com/package/websocket-stream)\n\n```Java\nimport io.appium.java_client.AppiumDriver;\nimport org.java_websocket.client.WebSocketClient;\nimport org.java_websocket.handshake.ServerHandshake;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport java.net.MalformedURLException;\nimport java.net.URI;\nimport java.net.URISyntaxException;\nimport java.net.URL;\n\n\n\npublic class Edition055_ADB_Logcat_Streaming {\n\n private String ANDROID_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk\";\n private String IOS_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.6.1/TheApp-v1.6.1.app.zip\";\n\n private AppiumDriver driver;\n\n public class LogClient extends WebSocketClient {\n\n public LogClient( URI serverURI ) {\n super(serverURI);\n }\n\n @Override\n public void onOpen(ServerHandshake handshakedata) {\n System.out.println(\"WEBSOCKET OPENED\");\n }\n\n @Override\n public void onMessage(String message) {\n System.out.println(message);\n }\n\n @Override\n public void onClose(int code, String reason, boolean remote) {\n System.out.println(\"Connection closed, log streaming has stopped\");\n }\n\n @Override\n public void onError(Exception ex) {\n ex.printStackTrace();\n // if the error is fatal then onClose will be called additionally\n }\n }\n\n @Test\n public void streamAndroidLogs() throws URISyntaxException, MalformedURLException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"app\", ANDROID_APP);\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n\n LogClient logClient = new LogClient(new URI( \"ws://localhost:4723/ws/session/\" + driver.getSessionId() + \"/appium/device/logcat\"));\n\n driver.executeScript(\"mobile:startLogsBroadcast\");\n\n logClient.connect();\n\n try { Thread.sleep(5000); } catch (Exception ign) {} // logs printed to stdout while we're sleeping.\n\n driver.executeScript(\"mobile:stopLogsBroadcast\");\n driver.quit();\n }\n\n @Test\n public void streamIOSLogs() throws URISyntaxException, MalformedURLException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.1\");\n caps.setCapability(\"deviceName\", \"iPhone XS\");\n caps.setCapability(\"automationName\", \"XCUITest\");\n caps.setCapability(\"app\", IOS_APP);\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n\n LogClient logClient = new LogClient( new URI(\"ws://localhost:4723/ws/session/\" + driver.getSessionId() + \"/appium/device/syslog\"));\n\n driver.executeScript(\"mobile:startLogsBroadcast\");\n\n logClient.connect();\n\n try { Thread.sleep(5000); } catch (Exception ign) {} // logs printed to stdout while we're sleeping.\n\n driver.executeScript(\"mobile:stopLogsBroadcast\");\n driver.quit();\n }\n}\n```\n\nAnd the Javascript example:\n\n```javascript\nlet test = require('ava')\nlet { remote } = require('webdriverio')\nlet B = require('bluebird')\nlet websocket = require('websocket-stream')\n\nlet driver\n\ntest('stream Android logcat logs', async t => {\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'Android',\n deviceName: 'Android Emulator',\n automationName: 'UiAutomator2',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk'\n },\n logLevel: 'error'\n })\n\n let logStream = websocket(`ws://localhost:4723/ws/session/${driver.sessionId}/appium/device/logcat`)\n logStream.pipe(process.stdout)\n\n logStream.on('finish', () => {\n console.log('Connection closed, log streaming has stopped')\n })\n\n driver.executeScript('mobile:startLogsBroadcast', [])\n\n await B.delay(5000)\n\n driver.executeScript('mobile:stopLogsBroadcast', [])\n\n await driver.deleteSession()\n t.pass()\n})\n\ntest('stream iOS system logs', async t => {\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'iOS',\n platformVersion: '12.1',\n deviceName: 'iPhone XS',\n automationName: 'XCUITest',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.6.1/TheApp-v1.6.1.app.zip'\n },\n logLevel: 'error'\n })\n\n let logStream = websocket(`ws://localhost:4723/ws/session/${driver.sessionId}/appium/device/syslog`)\n logStream.pipe(process.stdout)\n\n logStream.on('finish', () => {\n console.log('Connection closed, log streaming has stopped')\n })\n\n driver.executeScript('mobile:startLogsBroadcast', [])\n\n await B.delay(5000)\n\n driver.executeScript('mobile:stopLogsBroadcast', [])\n\n await driver.deleteSession()\n t.pass()\n})\n```\n\n(Both versions of the full code demonstration are available [on GitHub](https://github.com/cloudgrey-io/appiumpro/))\n","metadata":{"lang":"java","platform":"All Platforms","subPlatform":"All Devices","title":"Using Mobile Execution Commands to Continuously Stream Device Logs with Appium","shortDesc":"Using the 'mobile:starLogsBroadcast' command, we can open a WebSocket connection to Appium in order to asynchronously stream device logs on both Android and iOS","liveAt":"2019-02-06 10:00","canonicalRef":"https://www.headspin.io/blog/using-mobile-execution-commands-to-continuously-stream-device-logs-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/55-using-mobile-execution-commands-to-continuously-stream-device-logs-with-appium","slug":"55-using-mobile-execution-commands-to-continuously-stream-device-logs-with-appium","path":"/editions/55-using-mobile-execution-commands-to-continuously-stream-device-logs-with-appium","news":{"rawMd":"While working on my upcoming talk for [AppiumConf](https://appiumconf.com/) in which I will attempt to expound upon every Desired Capability supported by Appium, I have already found a dozen desired capabilities which were undocumented. Among them is `clearDeviceLogsOnStart`, which will truncate the device logs starting from when a session was started. I've added a mention of this to the [edition on accessing logcat logs](https://appiumpro.com/editions/53) from two weeks ago.\n\n(As part of my talk, I'll also be updating the documentation for desired capabilities.)\n","num":55,"mdPath":"/vercel/path0/content/news/0055.md"},"tags":["java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0054.md","num":54,"rawMd":"\n\nDuring the development of your test suite, it's helpful to run tests one at a time, and on one target platform at a time. But when it comes to a production CI environment, you're definitely going to want to run your tests in parallel, with as high a concurrency as you can manage. We've already looked at [how to run your Appium tests in parallel](https://appiumpro.com/editions/28), and it involves setting up specific capabilities in your test to make sure that, for each test, Appium can instantiate an appropriate driver without stepping on the toes of the drivers running _other_ tests.\n\n### Scaling Up Parallel Testing\n\nThis approach only scales so far, however. There's a limit to how many real or virtual devices you can run on one machine. What if you want to run 100 Appium tests in parallel? You're likely not going to be able to do that all with one Appium server. Or what if you're running a web test, and would like to make sure your testing works on desktop browsers as well as mobile devices? For that you'll need to mix in Selenium. Either way, you're eventually going to run into a situation where you need multiple automation servers running, which adds a lot of complexity to your test script. Now your test script needs to know the host and port of each server, which servers host which devices or browsers, and how to make sure it doesn't start too many tests on one server and overload its capacity. Essentially, you're dealing with an execution model that looks like this:\n\n

\n \"Parallel\n

\n\n(In this example, we're trying to run a web test in parallel across Safari on iOS, Chrome on Android, and Chrome on Desktop).\n\n### Selenium Grid\n\nThankfully, there's a way to mitigate all this complexity, by using a special-purpose [load balancer](https://en.wikipedia.org/wiki/Load_balancing_(computing) called [Selenium Grid](https://github.com/SeleniumHQ/selenium/wiki/Grid2). Selenium Grid is basically a proxy server that speaks the WebDriver protocol, and manages connections to a pool of WebDriver servers. It wasn't designed specifically to work with Appium, but because Appium also speaks the WebDriver protocol (and because we taught Appium how to register with Selenium Grid), it's the best tool for DIY Appium clusters.\n\nHere's how Selenium Grid works:\n\n1. You start a Selenium server running in \"hub\" mode on a certain host and port of your choosing.\n2. Now, you can start any number of other Selenium or Appium servers in \"node\" mode. You can start them anywhere you like, as long as they have network access to the grid server (the \"hub\").\n3. Each node is started with a special \"node config\" that describes the kinds of automation capabilities it supports.\n4. Now, when your test needs to start a session, you ensure it targets the grid server, and the grid server will decide which node to use for the session. You don't need to know or care about where the nodes are located. As long as your test specifies capabilities which the hub can match to a free node, your test will run.\n\nIn other words, I've been describing a model that looks more like this:\n\n

\n \"Parallel\n

\n\nBeyond the simplicity of needing to know only about one test server, the Selenium Grid model is particularly nice for Appium, because Appium drivers also require a number of system resources (unique ports, etc...). On the Grid model, you can start each Appium server independently, with a predefined set of ports or the like, using the `--default-capabilities` flag. So, for example, I could start a set of Appium servers like this, also using the `--nodeconfig` flag to tell Appium it should connect with Selenium Grid:\n\n```\nappium --default-capabilities='{\"wdaLocalPort\": 8100}' --nodeconfig=/path/to/nodeconfig.json\nappium --default-capabilities='{\"wdaLocalPort\": 8200}' --nodeconfig=/path/to/nodeconfig.json\n```\n\nNow each of these Appium servers will be registered with the Selenium Grid, and each server has declared which `wdaLocalPort` it needs; now my test script doesn't have to know or care about this! It becomes an ops step, which makes much more sense.\n\n### Node Config\n\nThe magic pretty much all happens in the \"node config\" file, which is defined as JSON. There are a lot of [nodeconfig options](https://github.com/SeleniumHQ/selenium/wiki/Grid2#configuring-the-nodes) to choose from, but here's an example of the config I'll be using for the Safari-on-iOS node in this guide:\n\n```json\n{\n \"capabilities\":\n [\n {\n \"deviceName\": \"iPhone 8\",\n \"platformVersion\": \"11.4\",\n \"maxInstances\": 1,\n \"platformName\": \"iOS\",\n \"browserName\": \"Safari\"\n }\n ],\n \"configuration\":\n {\n \"cleanUpCycle\":2000,\n \"timeout\":30000,\n \"proxy\": \"org.openqa.grid.selenium.proxy.DefaultRemoteProxy\",\n \"maxSession\": 1,\n \"register\": true,\n \"registerCycle\": 5000,\n \"hubPort\": 4444,\n \"hubHost\": \"127.0.0.1\",\n \"hubProtocol\": \"http\"\n }\n}\n```\n\nLet's take a look at some of the important configuration keys:\n\n* `capabilities`: the set of capabilities supported by this node. Can be a list. In this case, the node is only supporting one type of device. Note that in addition to the capabilities you're used to seeing, you can specify how many of a particular capability set the node should be allowed to start, via `maxInstances`.\n* `proxy`: the proxy logic to be used for communicating with this node. You can write a custom proxy if desired.\n* `maxSession`: the total number of sessions this node can run. This is useful in combination with `maxInstances`. For example, let's say we can run a total of 5 iPhone 8s and 5 iPhone 6s on this node, but the system gets overloaded when running more than 8 sessions at a time, of _any_ kind. Then we'd set `maxInstances` to `5` for each device type, but `maxSession` to `8`, so that we never try to run the theoretical max of 10.\n* `hubHost`, `hubPort`, and `hubProtocol`: the connection details for the Grid hub server\n\n### Starting Up the Grid\n\nThat's basically all you need to know about how Selenium Grid works with Appium. So let's dive into a working example. What we want to end up with is parallel execution of one web test, across Safari on iOS, Chrome on Android, and Chrome on Desktop (just like in the diagrams above).\n\n1. Make sure you've downloaded [the latest Selenium server](https://www.seleniumhq.org/download/) (I'm on 3.141.59 for this guide).\n2. Make sure you've got the latest [Chromedriver](http://chromedriver.chromium.org/downloads), so that Chrome desktop automation works (I'm on 2.45).\n3. Make sure you've got an Android emulator up and running with the latest version of the Chrome browser on it (I'm using [v71](https://www.apkmirror.com/apk/google-inc/chrome/chrome-71-0-3578-99-release/google-chrome-fast-secure-71-0-3578-99-8-android-apk-download/download/)).\n4. Open up 4 separate terminal windows (we're going to run the Grid hub server in one, and the 3 nodes in the others). Here's what to run in the 4 windows (in all cases, replace the paths with appropriate paths for your system):\n\n```bash\n# Window 1: the Grid hub\njava -jar /path/to/selenium-server-standalone.jar -role hub\n\n# Window 2: the iOS node\nappium -p 4723 --nodeconfig /path/to/nodeconfig-ios.json\n\n# Window 3: the Android node\nappium -p 4733 --nodeconfig /path/to/nodeconfig-android.json\n\n# Window 4: the Chrome desktop node\njava -Dwebdriver.chrome.driver=\"/path/to/chromedriver\" -jar /path/to/selenium-server-standalone.jar -role node -nodeConfig /path/to/nodeconfig-chrome.json\n```\n\nHere are the contents of the nodeconfig JSON files (in addition to `nodeconfig-ios.json`, which is above). (You can also download them [from the Appium Pro repo on GitHub](https://github.com/cloudgrey-io/appiumpro/tree/master/java/src/test/java/Edition054_Selenium_Grid)).\n\n`nodeconfig-android.json`:\n\n```json\n{\n \"capabilities\":\n [\n {\n \"browserName\": \"Chrome\",\n \"platformVersion\": \"26\",\n \"maxInstances\": 1,\n \"platformName\": \"Android\",\n \"automationName\": \"UiAutomator2\",\n \"deviceName\": \"Android Emulator\"\n }\n ],\n \"configuration\":\n {\n \"cleanUpCycle\":2000,\n \"timeout\":30000,\n \"proxy\": \"org.openqa.grid.selenium.proxy.DefaultRemoteProxy\",\n \"maxSession\": 1,\n \"register\": true,\n \"registerCycle\": 5000,\n \"hubPort\": 4444,\n \"hubHost\": \"127.0.0.1\",\n \"hubProtocol\": \"http\"\n }\n}\n```\n\n`nodeconfig-chrome.json`:\n\n```json\n{\n \"capabilities\":\n [\n {\n \"browserName\": \"chrome\",\n \"browserVersion\": \"71\",\n \"platformName\": \"MAC\",\n \"maxInstances\": 1\n }\n ],\n \"cleanUpCycle\":2000,\n \"timeout\":30000,\n \"proxy\": \"org.openqa.grid.selenium.proxy.DefaultRemoteProxy\",\n \"maxSession\": 1,\n \"register\": true,\n \"registerCycle\": 5000,\n \"hubPort\": 4444,\n \"hubHost\": \"127.0.0.1\",\n \"hubProtocol\": \"http\"\n}\n```\n\nThis is all the setup we need! In a real production CI environment, you'd probably have this all automated and potentially registering lots more nodes across many different physical or virtual servers. Anyway, once you've got everything up and running, you should see messages in the Selenium Grid console to the effect that several nodes have checked in:\n\n```\n13:48:04.514 INFO [DefaultGridRegistry.add] - Registered a node http://0.0.0.0:4733\n```\n\n### Running Tests on the Grid\n\nBecause our test project is Gradle-based, our best option for running in parallel is to use `maxParallelForks` (as described in [Edition 28](https://appiumpro.com/editions/28), along with instructions for Maven-based projects). Here's how I've updated my `build.gradle` to that end:\n\n```gradle\ntest {\n maxParallelForks = 3\n forkEvery = 1\n}\n```\n\nThis will cause Gradle to create a new thread for each test class. Unfortunately, a consequence is that for real parallelism, we need to have separate test classes, not just separate methods within one test. For that reason, this example includes 4 Java files: one as a base test class for shared code, and 3 platform-specific classes, so that they can be run in parallel. Here's the base class:\n\n```java\npublic class Edition054_Selenium_Grid {\n protected RemoteWebDriver driver;\n\n static String HUB_URL = \"http://localhost:4444/wd/hub\";\n\n private static By loginBtn = By.tagName(\"button\");\n private static By username = By.id(\"username\");\n private static By password = By.id(\"password\");\n private static By message = By.id(\"flash\");\n\n private static String LOGIN_URL = \"https://the-internet.herokuapp.com/login\";\n private static String ERR_MSG = \"Your username is invalid!\";\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n protected void actualTest(RemoteWebDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n driver.navigate().to(LOGIN_URL);\n wait.until(ExpectedConditions.presenceOfElementLocated(username))\n .sendKeys(\"foo\");\n driver.findElement(password).sendKeys(\"bar\");\n driver.findElement(loginBtn).click();\n\n String errText = wait.until(ExpectedConditions.presenceOfElementLocated(message)).getText();\n Assert.assertThat(errText, Matchers.containsString(ERR_MSG));\n }\n}\n```\n\nAs you can see, it contains the actual test logic we want to execute, which attempts a login on [The Internet](https://the-internet.herokuapp.com) and checks that the login fails. The only real item of interest here is that we have saved a reference to the Grid hub server, running on the default port `4444`. This is what each test class will use to connect to the Grid hub.\n\nAnd here are the three platform-specific classes:\n\n\n```java\npublic class Edition054_Selenium_Grid_IOS extends Edition054_Selenium_Grid {\n\n @Test\n public void testIOS() throws MalformedURLException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"11.4\");\n caps.setCapability(\"deviceName\", \"iPhone 8\");\n caps.setCapability(\"browserName\", \"Safari\");\n\n driver = new RemoteWebDriver(new URL(HUB_URL), caps);\n actualTest(driver);\n }\n}\n\n\npublic class Edition054_Selenium_Grid_Android extends Edition054_Selenium_Grid {\n\n @Test\n public void testAndroid() throws MalformedURLException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"browserName\", \"Chrome\");\n\n driver = new RemoteWebDriver(new URL(HUB_URL), caps);\n actualTest(driver);\n }\n}\n\npublic class Edition054_Selenium_Grid_Desktop extends Edition054_Selenium_Grid {\n\n @Test\n public void testDesktop() throws MalformedURLException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"browserName\", \"chrome\");\n caps.setCapability(\"browserVersion\", \"71\");\n caps.setCapability(\"platformName\", \"MAC\");\n\n driver = new RemoteWebDriver(new URL(HUB_URL), caps);\n actualTest(driver);\n }\n}\n```\n\nThe only responsibility for these classes is to get the right capabilities set up, instantiate a driver with them, and kick off the actual test. With each of these classes set up in their own files appropriately, I can now use a `gradle` command to run my test suite in parallel:\n\n```\n./gradlew :cleanTest :test --tests \"Edition054_Selenium_Grid.Edition054_Selenium_Grid_*\"\n```\n\n(The `*` helps me select all the test classes I care about for this example, and not other ones).\n\nAt this point, all three tests are started simultaneously, and Selenium Grid forwards each session to the appropriate node. The result is that all the tests execute at the same time (though on my machine, the Chrome Desktop test ends much faster than the other two)!\n\n### Wrap-up\n\nSelenium Grid, and distributed parallel testing in general, is a deep topic, and this guide could only cover the basic cases. But what we've shown is enough to use to get a test grid of basically any size up and running, potentially even mixing and matching mobile and desktop testing as well.\n\nWhat you'll no doubt find as you get into setting up your own Grid is that, fabulous as Selenium Grid itself may be, there's an awful lot of maintenance associated with running the various nodes! That's one reason that there are, nowadays, a number of options for running your Appium tests in parallel in the cloud, and it's important to try solutions like these out before going all-in on an in-house Grid setup, so you can do appropriate cost-benefit analysis for your team.\n","metadata":{"lang":"java","platform":"All Platforms","subPlatform":"All Devices","title":"Using Appium With Selenium Grid","shortDesc":"Running your Appium tests in parallel is great, but using one Appium server doesn't scale to meet the needs of a production CI environment. For that, the best DIY option is Selenium Grid, which is a WebDriver-specific load balancer and proxy. Using Selenium Grid, you can easily add and remove capacity from your test grid, and mix in Selenium-based testing seamlessly alongside your Appium mobile tests.","liveAt":"2019-01-30 10:00","canonicalRef":"https://www.headspin.io/blog/using-appium-with-selenium-grid"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/54-using-appium-with-selenium-grid","slug":"54-using-appium-with-selenium-grid","path":"/editions/54-using-appium-with-selenium-grid","news":null,"tags":["java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0053.md","num":53,"rawMd":"\n\nAn Android device or emulator records detailed system logs, which can sometimes contain information which is pertinent to a test. Maybe our test failed and we want to see if our app logged an error to the system. Or maybe the only way to assert that our test succeeds is to make sure a particular pattern appears in the logs.\n\nAndroid systems make these logs available using a tool called [logcat](https://developer.android.com/studio/command-line/logcat), and Appium clients can access these logs remotely.\n\nBoth Appium and Selenium respond to the following JSON Wire Protocol endpoints:\n```\nGET /wd/hub/session/:sessionId/log/types\nPOST /wd/hub/session/:sessionId/log\n```\nWhich can be called using the following methods in the Java client:\n```java\ndriver.manage().logs().getAvailableLogTypes();\ndriver.manage().logs().get(\"logcat\");\n```\nThe Python client and WebdriverIO call the same methods this way:\n```python\ndriver.log_types # note this is a property, not a function\ndriver.get_log('logcat')\n```\n```javascript\ndriver.getLogTypes()\ndriver.getLogs('logcat')\n```\n\nThe first method returns a collection of Strings which represent the logs which Appium provides. We are looking for `\"logcat\"` logs, but notice that Appium also provides [android bug report](https://developer.android.com/studio/debug/bug-report) data.\n\nThe logs are returned as a `LogEntries` class which implements the `java.lang.Iterable` interface.\nA single `LogEntry` object has `toString()` and `toJson()` methods, as well as some methods to get individual parts of the message such as timestamp or log level.\n\nSomething interesting to note is that every call to `logs().get(\"logcat\")` will only return log entries since the last time `logs().get(\"logcat\")` was called. The beginning of the system logs are only returned on the first call to `logs().get(\"logcat\")`. The example code at the end of this post demonstrates this.\n\nIf we are running many tests over and over on the same device, it could be difficult to locate the point in the logs where our test ran (and maybe failed). If we wanted to address this, one strategy could be to call `logs().get(\"logcat\")` at the beginning of each test. That way, when code inside the test gets logs, only logs which were recorded after the test started will be included.\n```java\n@Before\npublic void consumeIrrelevantLogs() {\n driver.manage().logs().get(\"logcat\");\n}\n```\n\nThere is also a desired capability we can set which clears the logcat logs at the start of a session.\n```java\ndriver.setCapability(\"clearDeviceLogsOnStart\", true);\n```\n\nThe following example code shows how we can get the supported log types for our test session.\nThen we get all logcat logs, printing the first and last lines. We wait 5 seconds and then print the first ten lines returned again, demonstrating that instead of starting from the beginning of the system log, the second call resumes from where the first left off.\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.logging.LogEntries;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.Set;\nimport java.util.stream.StreamSupport;\n\npublic class Edition053_ADB_Logcat {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk\";\n\n private AppiumDriver driver;\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"app\", APP);\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void captureLogcat() {\n // inspect available log types\n Set logtypes = driver.manage().logs().getAvailableLogTypes();\n System.out.println(\"suported log types: \" + logtypes.toString()); // [logcat, bugreport, server, client]\n\n // print first and last 10 lines of logs\n LogEntries logs = driver.manage().logs().get(\"logcat\");\n System.out.println(\"First and last ten lines of log: \");\n StreamSupport.stream(logs.spliterator(), false).limit(10).forEach(System.out::println);\n System.out.println(\"...\");\n StreamSupport.stream(logs.spliterator(), false).skip(logs.getAll().size() - 10).forEach(System.out::println);\n\n // wait for more logs\n try { Thread.sleep(5000); } catch (Exception ign) {} // pause to allow visual verification\n\n // demonstrate that each time get logs, we only get new logs\n // which were generated since the last time we got logs\n LogEntries secondCallToLogs = driver.manage().logs().get(\"logcat\");\n System.out.println(\"\\nFirst ten lines of next log call: \");\n StreamSupport.stream(secondCallToLogs.spliterator(), false).limit(10).forEach(System.out::println);\n\n Assert.assertNotEquals(logs.iterator().next(), secondCallToLogs.iterator().next());\n }\n}\n```\n\nThis edition of AppiumPro also comes with Python and Javascript examples:\n\nPython:\n```python\nimport unittest\nimport time\nfrom appium import webdriver\n\nclass Edition053_ADB_Logcat(unittest.TestCase):\n\n def setUp(self):\n desired_caps = {}\n desired_caps['platformName'] = 'Android'\n desired_caps['automationName'] = 'UiAutomator2'\n desired_caps['deviceName'] = 'Android Emulator'\n desired_caps['app'] = 'https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk'\n\n self.driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps)\n\n def tearDown(self):\n # end the session\n self.driver.quit()\n\n def test_capture_logcat(self):\n # inspect available log types\n logtypes = self.driver.log_types\n print(' ,'.join(logtypes)) #\n\n # print first and last 10 lines of logs\n logs = self.driver.get_log('logcat')\n log_messages = list(map(lambda log: log['message'], logs))\n print('First and last ten lines of log: ')\n print('\\n'.join(log_messages[:10]))\n print('...')\n print('\\n'.join(log_messages[-9:]))\n\n # wait for more logs\n time.sleep(5)\n\n # demonstrate that each time get logs, we only get new logs\n # which were generated since the last time we got logs\n logs = self.driver.get_log('logcat')\n second_set_of_log_messages = list(map(lambda log: log['message'], logs))\n print('\\nFirst ten lines of second log call: ')\n print('\\n'.join(second_set_of_log_messages[:10]))\n\n assert log_messages[0] != second_set_of_log_messages[0]\n\nif __name__ == '__main__':\n suite = unittest.TestLoader().loadTestsFromTestCase(Edition053_ADB_Logcat)\n unittest.TextTestRunner(verbosity=2).run(suite)\n```\n\nJavascript:\n```javascript\nlet test = require('ava')\nlet { remote } = require('webdriverio')\nlet B = require('bluebird')\n\nlet driver\n\ntest.before(async t => {\n driver = await remote({\n hostname: 'localhost',\n port: 4723,\n path: '/wd/hub',\n capabilities: {\n platformName: 'Android',\n deviceName: 'Android Emulator',\n automationName: 'UiAutomator2',\n app: 'https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk'\n },\n logLevel: 'error'\n })\n})\n\ntest.after(async t => {\n await driver.deleteSession()\n})\n\ntest('capture logcat', async t => {\n // inspect available log types\n let logtypes = await driver.getLogTypes()\n console.log('supported log types: ', logtypes) // [ 'logcat', 'bugreport', 'server' ]\n\n // print first and last 10 lines of logs\n let logs = await driver.getLogs('logcat')\n console.log('First and last ten lines of logs:')\n console.log(logs.slice(0, 10).map(entry => entry.message).join('\\n'))\n console.log('...')\n console.log(logs.slice(-10).map(entry => entry.message).join('\\n'))\n\n // wait for more logs\n await B.delay(5000)\n\n // demonstrate that each time get logs, we only get new logs\n // which were generated since the last time we got logs\n let secondCallToLogs = await driver.getLogs('logcat')\n console.log('First ten lines of next log call: ')\n console.log(secondCallToLogs.slice(0, 10).map(entry => entry.message).join('\\n'))\n\n t.true(logs[0].message !== secondCallToLogs[0].message)\n})\n```\n\n(All three versions of the full code demonstration are available [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test))\n","metadata":{"lang":"java","platform":"Android","subPlatform":"All Devices","title":"Accessing Android Logcat Logs with Appium","shortDesc":"Android devices record detailed system logs, commonly referred to as 'logcat' logs. These logs sometimes contain information pertinent to tests, and can be easily read and saved from remote devices through the various Appium clients.","liveAt":"2019-01-23 10:00","canonicalRef":"https://www.headspin.io/blog/accessing-android-logcat-logs-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/53-accessing-android-logcat-logs-with-appium","slug":"53-accessing-android-logcat-logs-with-appium","path":"/editions/53-accessing-android-logcat-logs-with-appium","news":{"rawMd":"What did you think about the multi-language version of this edition of Appium Pro? [Let us know](https://appiumpro.com/contact)!\n\nAlso, this letter is Appium Pro edition 53, which means that we've now marked the first full year of Appium Pro! Thank you to all our readers, and thanks for your thoughtful comments, appreciation, and suggestions that keep us going forward. Thanks also to our amazing sponsors--[HeadSpin](https://headspin.io) and [Applitools](https://applitools.com)--who have faithfully supported this endeavor.\n\nSpeaking of progress, I'm excited to have Jonah Stiennon, whom you'll recognize from authoring several recent Appium Pro articles, [joining me at Cloud Grey](https://cloudgrey.io/blog/jonah-stiennon-joins-cloud-grey)! We're excited to tackle together the huge list of Appium Pro newsletter ideas---definitely enough for Year 2 of this project. So stay tuned for more great things from Appium Pro and Cloud Grey in 2019!\n","num":53,"mdPath":"/vercel/path0/content/news/0053.md"},"tags":["java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0052.md","num":52,"rawMd":"\n\nAppium isn't limited to automating mobile systems! As long as there is an open way to interact with a system, a driver can be written for it, and included in Appium. Using a project called [Appium For Mac](https://github.com/appium/appium-for-mac) Appium can automate native macOs apps.\n\n### Setup ###\n\nAppium comes bundled with a macOs driver, but the actual AppiumForMac binary is not included, so we need to install it ourselves first:\n\n* Start by downloading the latest release from [here](https://github.com/appium/appium-for-mac/releases).\n* Unzip the `AppiumForMac.zip` file by double-clicking it in Finder.\n* Move `AppiumForMac.app` file into your Applications folder.\n\nAppiumForMac uses the system Accessibility API in order to automate apps. We need to give expanded permissions to AppiumForMac in order for it to work. Here are instructions from the [README](https://github.com/appium/appium-for-mac#installation):\n> Open System Preferences > Security & Privacy.\n> Click the Privacy tab.\n> Click Accessibility in the left hand table.\n> If needed, click the lock to make changes.\n> If you do not see AppiumForMac.app in the list of apps, then drag it to the list from Finder.\n> Check the checkbox next to AppiumForMac.app.\n\n

\n \"Enable\n

\n\nI'm using the latest version of macOS (10.14.2), if you are using an earlier version, specific instructions are included in the [AppiumForMac Readme](https://github.com/appium/appium-for-mac#installation).\n\n### Run a Test ###\n\nThe [example code for AppiumForMac](https://github.com/appium/appium-for-mac/tree/master/examples/others) already automates the calculator app, so let's do something different and automate the Activity Monitor instead.\n\nIn order to automate a macOs app, we only need to set the following desired capabilities:\n\n```json\n{\n \"platformName\": \"Mac\",\n \"deviceName\": \"Mac\",\n \"app\": \"Activity Monitor\"\n}\n```\n\nWe specify `Mac` as the platform and set `app` to the name of the installed app we want to run.\nOnce a test has been started, an app can also be launched using the `GET` command, e.g.:\n\n```java\ndriver.get(\"Calculator\")\n```\n\n### Absolute AXPath ###\n\nAppiumForMac is a little tricky, since elements can only be found using a special kind of XPath selector called \"absolute AXPath\". All the AXPath selectors use Accessibility API identifiers and properties. I'm including the exact rules for AXPath selectors below, but don't be afraid if they do not make sense at first; in the next section I describe some tools for finding AXPath selectors.\n\nHere are the rules for a valid Absolute AXPath selector:\n - Uses OS X Accessibility properties, e.g. AXMenuItem or AXTitle. You cannot use any property of an element besides these.\n - Must begin with `/AXApplication`.\n - Must contain at least one other node following `/AXApplication`.\n - Does not contain \"//\", or use a wildcard, or specify multiple paths using `|`.\n - Uses predicates with a single integer as an index, or one or more string comparisons using `=` and `!=`.\n - May use boolean operators `and` or `or` in between multiple comparisons, but may not include both `and` and `or` in a single statement. `and` and `or` must be surrounded by spaces.\n - Does not use predicate strings containing braces [] or parentheses ().\n - Uses single quotes, not double quotes for attribute strings.\n - Does not contain spaces except inside quotes and surrounding the `and` and `or` operators.\n\nAny XPath selector that follows the above rules will work as an absolute AXPath selector.\nBe warned: if your AXPath selector breaks the rules, you won't get a special error and instead will get an ElementNotFound exception. It can be difficult to identify whether your selectors are failing because the AXPath is invalid or the element simply is not on the screen.\n\nThe [README](https://github.com/appium/appium-for-mac#absolute-axpath) contains the following examples as guidance:\n\n```\nGood examples:\n\n\"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXGroup[1]/AXButton[@AXDescription='clear']\"\n\n\"/AXApplication[@AXTitle='Calculator']/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Scientific']\"\n\n\"/AXApplication/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Basic' and @AXMenuItemMarkChar!='']\"\n\nBad examples:\n\n\"//AXButton[@AXDescription='clear']\"\n(does not begin with /AXApplication, and contains //)\n\n\"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXButton[@AXDescription='clear']\"\n(not an absolute path: missing AXGroup)\n\n\"/AXApplication[@AXTitle=\"Calculator\"]/AXWindow[0]\"\n(a predicate string uses double quotes)\n\n\"/AXApplication[@AXTitle='Calculator']\"\n(path does not contain at least two nodes)\n\n\"/AXApplication[@AXTitle='Calculator']/AXMenuBar/AXMenuBarItems/AXMenuBarItem[@AXTitle='(Window)']\"\n(a predicate string contains forbidden characters)\n\n\"/AXApplication[@AXTitle='Calculator']/AXWindow[0]/AXGroup[1]/AXButton[@AXDescription ='clear']\"\n(a predicate contain a space before the =)\n\n\"/AXApplication[@AXTitle='Calculator']/AXWindow[position()>3]/AXGroup[1]/AXButton[@AXDescription='clear']\"\n(a predicate is not a simple string or integer, and specifies more than one node)\n\n\"/AXApplication/AXMenuBarItems/AXMenuBarItem[@AXTitle='View']/AXMenu/AXMenuItem[@AXTitle='Basic' and@AXMenuItemMarkChar!='']\"\n(leading and trailing spaces required for the boolean operator)\n\n\"/AXApplication[@AXTitle=\"Calculator\"]/AXWindow[0]/AXButton[@AXDescription='clear' and @AXEnabled='YES' or @AXDescription='clear all']\"\n(predicate uses multiple kinds of boolean operators; use one or more 'and', or, use one or more 'or', but not both)\n```\n\n### Tools for working with AXPath Selectors ###\n\nThis special AXPath restriction is tricky to work with, but we have some tools at our disposal.\n\nFirst of all, AppiumForMac provides a tool for generating the AXPath of any element on the screen. First, launch the AppiumForMac app manually using Finder or Launchpad. It won't display a window, but will appear in the dock. If you hold the `fn` key on your keyboard down for about three seconds, AppiumForMac will find the AXPath string to select whichever element your mouse pointer is currently hovering over. It stores the AXPath selector into your clipboard, so you can paste it into your test code. You'll know when it has worked because the AppiumForMac icon jumps out of the dock.\n\nThis behavior will work anywhere on your screen, because AppiumForMac can actually automate anything which is available to the Accessibility API.\n\n(NB: Third-party keyboards may not work with this functionality.)\n\nI found the AXPath strings generated by AppiumForMac to be pretty long. Make sure to organize your test so common parts of the string can be reused. I also removed many of the predicates since they were too-specific and not necessary.\n\nAnother tool which can help with AXPath strings is the [Accessiblity Inspector](https://developer.apple.com/library/archive/documentation/Accessibility/Conceptual/AccessibilityMacOSX/OSXAXTestingApps.html). This tool will show the hierarchy of accessibility elements, allow you to click on an element to inspect it, and view properties on elements.\n\nAs a last resort, you can try to dump the entire view hierarchy by calling `driver.getSource()`. This works on simple apps, but hung indefinitely on the Activity Monitor app, most likely because the UI is constantly updating.\n\n### The Test ###\n\nHere's an example test which starts the Activity Monitor, switches between tabs, and performs a search:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.concurrent.TimeUnit;\n\npublic class Edition052_Automate_Mac {\n\n private AppiumDriver driver;\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Mac\");\n caps.setCapability(\"deviceName\", \"Mac\");\n\n caps.setCapability(\"app\", \"Activity Monitor\");\n caps.setCapability(\"newCommandTimeout\", 300);\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testActivityMonitor() {\n String baseAXPath = \"/AXApplication[@AXTitle='Activity Monitor']/AXWindow\";\n String tabSelectorTemplate = baseAXPath + \"/AXToolbar/AXGroup/AXRadioGroup/AXRadioButton[@AXTitle='%s']\";\n driver.findElementByXPath(String.format(tabSelectorTemplate, \"Memory\")).click();\n driver.findElementByXPath(String.format(tabSelectorTemplate, \"Energy\")).click();\n driver.findElementByXPath(String.format(tabSelectorTemplate, \"Disk\")).click();\n driver.findElementByXPath(String.format(tabSelectorTemplate, \"Network\")).click();\n driver.findElementByXPath(String.format(tabSelectorTemplate, \"CPU\")).click();\n\n WebElement searchField = driver.findElementByXPath(baseAXPath + \"/AXToolbar/AXGroup/AXTextField[@AXSubrole='AXSearchField']\");\n searchField.sendKeys(\"Activity Monitor\");\n\n WebElement firstRow = driver.findElementByXPath(baseAXPath + \"/AXScrollArea/AXOutline/AXRow[0]/AXStaticText\");\n\n Assert.assertEquals(\" Activity Monitor\", firstRow.getText());\n }\n}\n```\n\n(As always, the full code sample is also up [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition052_Automate_Mac.java))\n\n### Caveats ###\n\nAppiumForMac is rough around the edges, probably because it does not currently have a lot of community use. Check the [Github issues](https://github.com/appium/appium-for-mac/issues) if you get stuck.\n\nNothing prevents AppiumForMac from being improved; contributions welcome!\n","metadata":{"lang":"java","platform":"MacOs","subPlatform":"All Devices","title":"Automating Mac Apps with Appium","shortDesc":"Appium isn't just for automating on mobile. The appium-mac-driver can automate native MacOs (OSX) applications, which showcases the generality of the Appium approach for functional UI automation.","liveAt":"2019-01-16 10:00","canonicalRef":"https://www.headspin.io/blog/automating-mac-apps-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/52-automating-mac-apps-with-appium","slug":"52-automating-mac-apps-with-appium","path":"/editions/52-automating-mac-apps-with-appium","news":null,"tags":["java","MacOs","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0051.md","num":51,"rawMd":"\n\nAppium is traditionally considered a \"black box\" testing tool, meaning it has no access to your application's internal methods or state. We use Appium correctly by thinking like a user would (interacting with the surface of the app), not thinking like an app developer would (calling internal code directly).\n\nBlack box testing has its limitations, primarily in requiring the automation to go through user steps many times, even when it would be more convenient to skip to a certain known state. This is one reason that some modern testing technologies, like Espresso, allow for a [white box testing](https://en.wikipedia.org/wiki/White-box_testing) approach, where internal app methods are accessible from the automation context.\n\nThanks to Appium's Espresso driver, Appium can now take advantage of this approach. (If you recall, Espresso is also what enabled us to [make our elements flash on screen](https://appiumpro.com/editions/48)). To make this more general white box strategy work, you need two things:\n\n1. Knowledge of a particular public method located on your Android application, activity, or UI element. This is the method that your test script will ultimately trigger in the course of your automation. You can either code this method up yourself or ping your Android app developer to add one that meets your specifications.\n2. The new `mobile: backdoor` method available on the Appium Espresso driver. This is the actual method you will call in your test code, which will tell the Espresso driver what to run inside your app. (It's called \"backdoor\" because Appium is getting inside of your app through the \"back door\" of Espresso, not the \"front door\" of the UI that a user would use. This approach was first publicly suggested by Rajdeep Varma in his [AppiumConf 2018 talk](https://www.youtube.com/watch?v=ap4Zp6X-sWc), and Rajdeep was the one who contributed the code to make it a reality in Appium today. Thanks!)\n\nTo illustrate how this works, I've added a method to the application class of [The App](https://github.com/cloudgrey-io/the-app), in its `MainApplication.java`:\n\n```java\npublic void raiseToast(String message) {\n Toast.makeText(this, message, Toast.LENGTH_LONG).show();\n}\n```\n\nThis method simply takes an arbitrary string and uses it to make an Android Toast message appear on the screen. The result of calling this method looks like this:\n\n

\n \"Hello,\n

\n\nWith a normal Appium test, there's no way I would be able to trigger this toast to appear, unless the developer had hooked it up to a text field and a button. But with `mobile: backdoor`, I can simply designate the name of the method I want to call in my app, the types and values of its parameters, and off we go! So here's how I would do this in Java:\n\n```java\nImmutableMap scriptArgs = ImmutableMap.of(\n \"target\", \"application\",\n \"methods\", Arrays.asList(ImmutableMap.of(\n \"name\", \"raiseToast\",\n \"args\", Arrays.asList(ImmutableMap.of(\n \"value\", \"Hello from the test script!\",\n \"type\", \"String\"\n ))\n ))\n);\n\ndriver.executeScript(\"mobile: backdoor\", scriptArgs);\n```\n\nIt's a little verbose, so let's look at the parameter I'm passing to `mobile: backdoor` as a JSON object instead:\n\n```json\n{\n \"target\": \"application\",\n \"methods\": [{\n \"name\": \"raiseToast\",\n \"args\": [{\n \"value\": \"Hello from the test script!\",\n \"type\": \"String\"\n }]\n }]\n}\n```\n\nI specify two main bits of information: the _target_ of my backdoor (which type of thing am I calling the method on), and the _methods_ I want to call on it. In this case I have implemented my method on the application class, so I specify `application` as the target. Other possible values are `activity` (for methods implemented on the current activity), or `element` (for methods implemented on a specific UI element--in this case, an `elementId` parameter is also required).\n\nThe most important information resides in `methods`. In our case I'm just calling one method, though we could call multiple. For each method, we have to specify its name (`raiseToast`--it must exactly match the name in my Android code), and the potentially multiple arguments we want to pass in to call the method with. For each of these arguments in turn we must specify both its type and value (the type is necessary because Java!).\n\nOnce we've got all this put together, we simply bundle it up and pass it as the parameter to `executeScript(\"mobile: backdoor\")`. Using the `ImmutableMap.of` construction as I've shown above is the most concise way I've found so far to do this in Java.\n\nThat's all there is to it! This method in conjunction with the Espresso driver frees us from the shackles of the UI, and enables us to target specific methods inside our app. The possibilities here are endless, so please write to us and let us know what cool things you find to do with the feature. But remember, with great power comes great responsibility! It would probably be very easy to use this feature to crash your app during testing, too.\n\nHere's a full example that shows the toast message being raised on our test app:\n\n```java\nimport com.google.common.collect.ImmutableMap;\nimport io.appium.java_client.AppiumDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.Arrays;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition051_Android_Backdoor {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.1/TheApp-v1.8.1.apk\";\n\n private AppiumDriver driver;\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"Espresso\");\n caps.setCapability(\"app\", APP);\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testBackdoor() {\n ImmutableMap scriptArgs = ImmutableMap.of(\n \"target\", \"application\",\n \"methods\", Arrays.asList(ImmutableMap.of(\n \"name\", \"raiseToast\",\n \"args\", Arrays.asList(ImmutableMap.of(\n \"value\", \"Hello from the test script!\",\n \"type\", \"String\"\n ))\n ))\n );\n\n driver.executeScript(\"mobile: backdoor\", scriptArgs);\n try { Thread.sleep(2000); } catch (Exception ign) {} // pause to allow visual verification\n }\n\n}\n```\n\n(As always, the full code sample is also up [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition051_Android_Backdoor.java))\n","metadata":{"lang":"java","platform":"Android","subPlatform":"All Devices","title":"Calling Methods Inside Your App From Appium","shortDesc":"Appium, combined with the power of Espresso, can finally break its way out of the black box testing model. Whether this is something you want to do is of course up to you, but using the mobile: backdoor method, you can trigger app-internal methods with arbitrary parameters.","liveAt":"2019-01-09 10:00","canonicalRef":"https://www.headspin.io/blog/calling-methods-inside-your-app-from-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/51-calling-methods-inside-your-app-from-appium","slug":"51-calling-methods-inside-your-app-from-appium","path":"/editions/51-calling-methods-inside-your-app-from-appium","news":null,"tags":["java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0050.md","num":50,"rawMd":"\n\n> Happy start of 2019! This is another post from Appium contributor Jonah Stiennon. In the past, I've labeled his contributions to Appium Pro \"guest posts\", but as of now, Jonah has officially joined [Cloud Grey](https://cloudgrey.io) as a partner! More to come about Jonah later, but meanwhile get ready to see more of him in this newsletter, on Appium's GitHub, and around the world consulting with you all.\n\nThanks to Appium contributor [@AnnaWyrwal](https://github.com/AnnaWyrwal), there are two new desired capabilities available in the latest Appium beta release (see our [earlier post](https://appiumpro.com/editions/47) on how to install beta versions of Appium). Under the right circumstances these can improve the time it takes to start an Android session.\n\n### `skipDeviceInitialization` ###\n\nAvailable for all Android platforms, the `skipDeviceInitialization` desired capability can be passed with the boolean value `true` to skip installing the [io.appium.settings app](https://github.com/appium/io.appium.settings) on the device. This special app is installed by the [Appium Android Driver](https://github.com/appium/appium-android-driver) at the beginning of each test and is used to manage specific settings on the device, such as:\n\n - wifi/data settings\n - disabling animation\n - setting a locale\n - IME settings\n\nWithout the io.appium.settings app, the Appium Android driver cannot automate these functions, but if the device previously had this app installed by Appium, it doesn't need to install it again. If you know that your device is already in the proper state then you can set `skipDeviceInitialization` to `true` and skip the time it takes to reinstall it.\n\nAppium already checks if the settings app is already installed on the device, but with this capability enabled it even skips the check to see if the app is installed.\n\nThis is most useful for tests which follow a previous successful test, or tests being run on the same emulators or devices over and over.\n\n### `skipServerInstallation` ###\n\nThe `skipServerInstallation` desired capability only applies when using the `UiAutomator2` automation method. The way the UIAutomator2 driver works, it installs a special [server](https://github.com/appium/appium-uiautomator2-server) onto the device, which listens to test commands from Appium and executes them.\n\nBy default, Appium installs this server onto the device at the beginning of every test session. If the value of `skipServerInstallation` is set to `true`, you can skip the time it takes to install this server.\n\nOf course, without the server on the device Appium can't automate it, but if you know that the server was installed during a previous test run you can safely skip this step.\n\nA warning: make sure to disable this capability if you have updated the version of Appium you are using, because you want it to install the latest version of the UIAutomator2 server.\n\nThis capability is also most useful for tests which follow a previous successful test, or tests being run on the same emulators or devices over and over.\n\n### Using `appPackage` and `appActivity` to launch existing apps ###\n\nAnother major time-saver when it comes to Android tests is using the `appPackage` and `appActivity` desired capabilities instead of the `app` capability.\n\nWe need to tell Appium which app to test. Usually we use the `app` capability, which can be a path to a `.apk`, `.apks`, or `.zip` file stored on your computer's filesystem or stored on a public website. If the app under test is known to already be installed on the device (most likely from a previous test run), the app package name and main activity can be passed instead (using `appPackage` and `appActivity` desired capabilities). Skipping the time to download a large file or install it on the Android operating system leads to big savings.\n\nAs with the previously discussed capabilities, make sure that you *do* install the app again if you are testing a newer version. Nothing is more frustrating than accidentally running tests on an older version of your app!\n\n### Honorary Mention: `ignoreUnimportantViews` ###\n\nThe `ignoreUnimportantViews` desired capability is not new, and is outside the scope of this post, but it deserves to be mentioned as another way to potentially speed up Android automation tests, especially if your tests focus on finding many elements using XPath locators. Set this to `true` to speed up Appium's ability to find elements in Android apps. (To learn more about this capability and other capabilities used for speeding up test execution, check out [Part 6 of the series on making your Appium tests fast and reliable](https://appiumpro.com/editions/24)).\n\nBelow is an example of the `skipDeviceInitialization`, `skipServerInstallation`, and `appPackage` desired capabilities being used to speed up an Android test. The first test runs without these capabilities in order to make sure that the io.appium.settings app and the UiAutomator2-server are installed. The second test runs without bothering to check or install these dependencies or the app under test on the device. By my not-so-rigorous analysis, `skipDeviceInitialization` and `skipServerInstallation` saved about 1 second per test and using `appPackage` saved ten seconds per test (mostly because the example code downloads an app from the internet).\n\n```java\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\nimport java.io.IOException;\nimport java.net.URL;\n\nimport io.appium.java_client.AppiumDriver;\n\npublic class Edition049_Faster_Android_Capabilities {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.0/TheApp-v1.8.0.apk\";\n\n private AppiumDriver driver;\n private WebDriverWait wait;\n\n @Test\n public void testInstallingDependencies() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n caps.setCapability(\"app\", APP);\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n\n // Test code goes here\n\n driver.quit();\n }\n\n @Test\n public void testSkippingInstallOfDependencies() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n\n // App is already installed on device, so can be launched by Package name and Activity\n caps.setCapability(\"appPackage\", \"io.cloudgrey.the_app\");\n caps.setCapability(\"appActivity\", \"io.cloudgrey.the_app.MainActivity\");\n\n // Skip the installation of io.appium.settings app and the UIAutomator 2 server.\n caps.setCapability(\"skipDeviceInitialization\", true);\n caps.setCapability(\"skipServerInstallation\", true);\n\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n\n // Test code goes here\n\n driver.quit();\n }\n}\n```\n","metadata":{"lang":"java","platform":"Android","subPlatform":"All Devices","title":"Special Capabilities for Speeding up Android Test Initialization","shortDesc":"Some new desired capabilities have been added, which when appropriately used can greatly increase the overall time it takes to run a test suite on Android devices.","liveAt":"2019-01-02 10:00","canonicalRef":"https://www.headspin.io/blog/special-capabilities-for-speeding-up-android-test-initialization"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/50-special-capabilities-for-speeding-up-android-test-initialization","slug":"50-special-capabilities-for-speeding-up-android-test-initialization","path":"/editions/50-special-capabilities-for-speeding-up-android-test-initialization","news":null,"tags":["java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0049.md","num":49,"rawMd":"\n\nAs this is the last Appium Pro edition of 2018, I thought it appropriate to take a few moments and reflect on what we're all doing here. Whether you celebrate Christmas, Hanukkah, Kwanzaa, Festivus, the Winter Solstice, or nothing in particular, the fact that you're reading my online newsletter about mobile app testing _probably_ means you have something to celebrate.\n\nI'm not saying life is easy for any of us, and I'm not saying that end-of-year holidays are always filled with joy--they can certainly be filled with heartbreak too, no matter how well-off we are. But I am saying that it's worth reflecting on the opportunities open to us in virtue of our experience with software development and/or testing. I'm grateful that I can spend a portion of every week thinking about fun and interesting things to do with Appium, and I'm even more grateful that there are some of you out there in the world who find these little guides worth reading. When I stop and think about the reality of how I get to live my life, I'm overwhelmed by the truth that there's nothing particularly \"fair\" about it.\n\nOK--this project is called \"Appium Pro\" and not \"Philosophical Ethics Pro\", so I won't belabor the point any further. I just wanted to give us all a brief moment to pause, detach from the technical details for a moment, think about what it means for us to be humans (even technologically-oriented humans), and have regard for those of our fellow human beings who are worried about much more urgent problems than how to solve the next automation challenge at work.\n\nIn that spirit, I offer you a different kind of code sample today, one that you _could_ use to donate to a variety of charities. I've chosen to write this little script to donate to [Raising the Roof](https://www.raisingtheroof.org/), a homelessness solutions charity here where I live in Canada. (The script will work for any charity using [gifttool.com](https://gifttool.com)'s platform, though it was not easy to find a list of such organizations). But find your favorite charity and write your own donation script! I just ran mine and it gave me a little thrill of holiday cheer. Let me know if you do something similar and I'll give you a shoutout in 2019!\n\nSee you all next year.\n\n```java\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.Select;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition049_Holidays_2018 {\n private IOSDriver driver;\n private WebDriverWait wait;\n\n private static String CHARITY_URL = \"https://www.gifttool.com/donations/Donate?ID=1676&AID=1607\";\n private static String DONATION_AMT = \"25\";\n private static String EMAIL = \"your@email.com\";\n private static String FIRST_NAME = \"Your\";\n private static String LAST_NAME = \"Name\";\n private static String STREET = \"123 Main Street\";\n private static String CITY = \"Vancouver\";\n private static String COUNTRY_CODE = \"CA\"; // 2-digit country code\n private static String STATE_CODE = \"BC\";\n private static String POSTAL_CODE = \"V6K 123\";\n private static String PHONE = \"555-555-5555\";\n private static String CARD_NAME = FIRST_NAME + \" \" + LAST_NAME;\n private static String CARD_NUMBER = \"1234 5678 9012 3456\";\n private static String CVV2 = \"123\";\n private static String CARD_TYPE = \"Visa\"; // 'Visa' or 'Mastercard'\n private static String CARD_MONTH = \"01\"; // 2-digit month\n private static String CARD_YEAR = \"21\"; // 2-digit year\n\n private static By donationAmt = By.id(\"ContributionAmount\");\n private static By donorEmail = By.name(\"Email\");\n private static By donorFirstName = By.name(\"FirstName\");\n private static By donorLastName = By.name(\"LastName\");\n private static By donorStreet = By.name(\"Street\");\n private static By donorCity = By.name(\"City\");\n private static By donorCountry = By.id(\"Country\");\n private static By donorState = By.id(\"State\");\n private static By donorPostalCode = By.name(\"ZipCode\");\n private static By donorPhone = By.name(\"PhoneDay\");\n private static By donorCardName = By.name(\"CardName\");\n private static By donorCardNumber = By.name(\"CardNumber\");\n private static By donorCVV2 = By.name(\"CVV2\");\n private static By donorCardType = By.name(\"CardType\");\n private static By donorCardMonth = By.name(\"ExpiryMonth\");\n private static By donorCardYear = By.name(\"ExpiryYear\");\n private static By submitDonation = By.id(\"gtSubmitButton\");\n\n @Before\n public void setUp() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 6\");\n capabilities.setCapability(\"browserName\", \"Safari\");\n\n driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testHolidayCheer() {\n driver.get(CHARITY_URL);\n wait.until(ExpectedConditions.presenceOfElementLocated(donationAmt)).sendKeys(DONATION_AMT);\n driver.findElement(donorEmail).sendKeys(EMAIL);\n driver.findElement(donorFirstName).sendKeys(FIRST_NAME);\n driver.findElement(donorLastName).sendKeys(LAST_NAME);\n driver.findElement(donorStreet).sendKeys(STREET);\n driver.findElement(donorCity).sendKeys(CITY);\n new Select(driver.findElement(donorCountry)).selectByValue(COUNTRY_CODE);\n new Select(driver.findElement(donorState)).selectByValue(STATE_CODE);\n driver.findElement(donorPostalCode).sendKeys(POSTAL_CODE);\n driver.findElement(donorPhone).sendKeys(PHONE);\n driver.findElement(donorCardName).sendKeys(CARD_NAME);\n driver.findElement(donorCardNumber).sendKeys(CARD_NUMBER);\n driver.findElement(donorCVV2).sendKeys(CVV2);\n new Select(driver.findElement(donorCardType)).selectByValue(CARD_TYPE);\n new Select(driver.findElement(donorCardMonth)).selectByValue(CARD_MONTH);\n new Select(driver.findElement(donorCardYear)).selectByValue(CARD_YEAR);\n driver.findElement(submitDonation).click();\n }\n}\n```\n\n(Full code sample [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition049_Holidays_2018.java))\n","metadata":{"lang":"java","platform":"All Platforms","subPlatform":"All Devices","title":"Donating Holiday Cheer Using Appium","shortDesc":"Did you know that you can use Appium to do all kinds of things, including scripting your holiday donation to your favorite charity? Here's a little example to get you inspired to think out of the box with using Appium as a force of good in the world.","liveAt":"2018-12-26 10:00","canonicalRef":"https://www.headspin.io/blog/donating-holiday-cheer-using-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/49-donating-holiday-cheer-using-appium","slug":"49-donating-holiday-cheer-using-appium","path":"/editions/49-donating-holiday-cheer-using-appium","news":null,"tags":["java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0048.md","num":48,"rawMd":"\n\nAs we near some long holiday time in many parts of the world, no doubt many of you are seeing twinkling and flashing lights around your neighborhoods. Appium has also recently learned how to make elements twinkle and flash! And this is not only a way to add sparkle to your test runs, but it can be a useful technique for debugging errors.\n\n

\n \"Twinkle,\n

\n\nIt can happen, for example, that when you find an element, you may not have a reference to the element you _think_ you have. This might especially be true in the case of XPath locators which aren't robust enough---the locator might still get you back an element, but for all you know it is now a completely different element.\n\nIn some cases, this would be apparent by getting the text of the element. In other cases, the only way to tell if you have the right element would be to change its visual appearance so you can manually inspect it. With Appium's [Espresso Driver](https://appium.io/docs/en/drivers/android-espresso/), this is now possible! How does this magic work? Because Espresso as a framework allows [white-box testing](https://en.wikipedia.org/wiki/White-box_testing), the Appium Espresso driver actually has access to the elements inside your app---not just an accessibility-powered representation of those elements.\n\nWe can therefore apply animations and other changes to elements that you've found, and this is precisely the point of the new `mobile: flashElement` command (available in the latest Appium beta). Basically, it takes an element reference and two other parameters, one for the duration of a single flash animation, and one for the number of times you wish the animation to be repeated. This gives you the flexibility you need to make sure the animation is visible in a video recorded of the session, or as you manually inspect your app.\n\nUsing the command is pretty simple. Let's say we have a `WebElement` object found and stored in a variable `el`. Then, we can do the typical `mobile:` method dance:\n\n```java\nHashMap scriptArgs = new HashMap<>();\nscriptArgs.put(\"element\", ((RemoteWebElement)el).getId());\nscriptArgs.put(\"durationMillis\", 50); // how long should each half-flash take?\nscriptArgs.put(\"repeatCount\", 20); // how many times should we flash?\n```\n\nIn this case, we will wind up with a series of quick flashes that takes about 2 seconds to complete. You can play around with these values to suit your liking.\n\nThat's all there is to it! This is a fun and moderately useful example of how leveraging Appium's new Espresso driver can make your life as a tester a tiny bit easier. As a full example, see the code below. Notice that I have included Android keystore-related capabilities; this is because I am using a signed release APK, and in this case it is a requirement that Appium's Espresso server and the test APK be signed with the same key. I thus need to tell Appium which key I used to sign my app so it can sign the Espresso server with it as well. If you're simply running a debug APK, you probably don't need to do this, but I include it because no doubt many of you will want to test release APKs.\n\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.remote.RemoteWebElement;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition048_Flash_Element {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.0/TheApp-v1.8.0.apk\";\n\n private AppiumDriver driver;\n private WebDriverWait wait;\n\n private By loginScreen = MobileBy.AccessibilityId(\"Login Screen\");\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"Espresso\");\n caps.setCapability(\"forceEspressoRebuild\", true);\n caps.setCapability(\"useKeystore\", true);\n caps.setCapability(\"keystorePath\", \"/Users/jlipps/.android/debug.keystore\"); // <-- replace with an appropriate path on your filesystem\n caps.setCapability(\"keystorePassword\", \"android\");\n caps.setCapability(\"keyAlias\", \"androiddebugkey\");\n caps.setCapability(\"keyPassword\", \"android\");\n\n\n caps.setCapability(\"app\", APP);\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testFlashElement() {\n WebElement el = wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen));\n\n HashMap scriptArgs = new HashMap<>();\n scriptArgs.put(\"element\", ((RemoteWebElement)el).getId());\n scriptArgs.put(\"durationMillis\", 50); // how long should each flash take?\n scriptArgs.put(\"repeatCount\", 20); // how many times should we flash?\n\n driver.executeScript(\"mobile: flashElement\", scriptArgs);\n }\n}\n```\n\nAnd as always, you can check out the code in the context of a working testsuite [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition048_Flash_Element.java).\n","metadata":{"lang":"java","platform":"Android","subPlatform":"All Devices","title":"Flashing Elements On Screen","shortDesc":"Add some sparkle to your Appium test by making elements flash! Not only a party trick, this can be a useful technique for debugging your test scripts to visually verify that you've got a handle on the elements you think you do!","liveAt":"2018-12-19 10:00","canonicalRef":"https://www.headspin.io/blog/flashing-elements-on-screen"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/48-flashing-elements-on-screen","slug":"48-flashing-elements-on-screen","path":"/editions/48-flashing-elements-on-screen","news":{"rawMd":"As you begin to wind down 2018 and look ahead to 2019, don't forget about [AppiumConf 2019](https://appiumconf.com)! And if you wanted to submit a proposal but didn't have a chance amidst year-end busyness, you may be in luck; there are rumours that the selection committee may be extending the deadline. So keep working on those ideas if you have them! If you're thinking at all of submitting, please submit whatever you have, as we work with submitters to help bring the best out of your idea. Or you can always e-mail me and I'm happy to comment on your proposal.\n","num":48,"mdPath":"/vercel/path0/content/news/0048.md"},"tags":["java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0047.md","num":47,"rawMd":"\n\n> This is another guest post from Appium contributor Jonah Stiennon. Enjoy!\n\nHave you run into a bug in Appium that has been fixed by open source contributors but hasn't been released in an official version yet? Or maybe you saw an exciting new feature you'd like to try out right away, rather than waiting for the next published release?\n\nIn order to do either of these, you can run Appium code which is newer than that latest release. Appium releases occur every one to three months. In between official versions, the Appium contributors release __beta__ versions, every week or so. You can install and use these beta versions, or for the very very latest code, you can run Appium from the most recent code posted to Github.\n\n### Installing and Running a Beta version\n\nAppium is a Node.js project hosted on npm. The latest beta version can be installed from the commandline as follows:\n```bash\nnpm uninstall -g appium\nnpm install -g appium@beta\n```\n\nAs easy as that. The first step is to remove a previously-installed version of Appium.\n\nIt should go without saying that a beta version of Appium may be less stable than an official release. Beta versions are not published without running tests first, but bugs may exist, especially in new features. Testers are encouraged to try use the beta versions, and [reporting bugs](https://github.com/appium/appium/issues) will ensure a better official release.\n\nAppium is composed of many modules, specifically one for each \"driver\" which controls mobile devices (xcuitest-driver, uiautomator2-driver, chrome-driver, safari-driver...). Most new code for Appium is added to these drivers rather than to the [Appium repository](https://github.com/appium/appium) itself. Whenever code is added to a driver, the driver is usually published to npm right away. Each time an official Appium release is created, specific versions of each driver are selected to be included in the Appium release.\n\nBeta releases of Appium do _not_ specify specific versions of drivers, so every time you install a beta version, all the drivers are updated to the latest published version. If you are already using a beta version of Appium, then all that is necessary to install newer versions of the drivers is to run:\n\n```bash\nnpm install -g appium@beta\n```\n\n### Installing Appium from Lastest Source Code\n\nAppium releases are published to npm, but the most recent code resides in github.\n\nFirst we clone the repository from github:\n\n```bash\ngit clone https://github.com/appium/appium.git\n```\n\nThen navigate to the new directory:\n```bash\ncd appium\n```\n\nThen install all Node.js dependencies:\n```bash\nnpm install\n```\n\nAppium requires some other dependencies you will need to install yourself, depending on which platforms you aim to automate. These are all covered in the [standard installation guides in the Appium docs](http://appium.io/docs/en/about-appium/getting-started/index.html) and aren't covered here.\n\nThe Appium code can be run as follows, within the Appium code directory:\n```bash\nnode .\n```\nWe can of course, choose to run this from anywhere else:\n```bash\nnode /path/to/appium/directory\n```\nAppium was not added to the PATH when we installed it, so running the `appium` command in a terminal will either fail, or run another version of Appium we have installed. We can set the `appium` command to run our source code by setting an alias. Add the following code to your shell config:\n\n```\nalias appium=\"node path/to/appium/directory\"\n```\n\nAs mentioned in the section about installing a beta version, the Appium project is comprised of many modules, especially all the various drivers for different platforms. When we ran `npm install` above, we installed the latest _published_ versions of those drivers, found on npm. Since we are running on the very latest code from github, we may also want to access the latest code for the one of the Appium drivers.\n\nIn order to have our local Appium source code use the latest code for a driver, we need to get the code for the driver from github. Then we use the [npm link](https://medium.com/@alexishevia/the-magic-behind-npm-link-d94dcb3a81af) command to tell Appium to use this code instead of a package from npm.\n\nAs an example, here is how we'd have our local Appium code use the latest code for the UiAutomator2 driver:\n\n```bash\ngit clone https://github.com/appium/appium-uiautomator2-driver.git\ncd appium-uiautomator2-driver\nnpm install\nnpm link\ncd /path/to/appium/directory\nnpm link appium-uiautomator2-driver\n```\n\nWe only need to set this up once, now if get pull the latest code into the appium-uiautomator2-driver directory (or modify the code there), restarting the Appium server will automatically include the new code.\n\nA warning though, if we install a new version of Appium or run `npm install` it will break this `npm link` and we'll have to do it again.\n\n### Installing an Old Version of Appium\n\nWhile we're on the subject, you can also install and run old versions of Appium like so:\n\n```bash\nnpm install appium@1.9.2\n```\nSubstitute the version you desire instead of `1.9.2`.\n","metadata":{"lang":"bash","platform":"All Platforms","subPlatform":"All Devices","title":"Running Appium From Source (Or The Latest Beta)","shortDesc":"Need to use Appium fixes and features which haven't been published yet? Run from the latest source code","liveAt":"2018-12-12 10:00","canonicalRef":"https://www.headspin.io/blog/running-appium-from-source-or-the-latest-beta"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/47-running-appium-from-source-or-the-latest-beta","slug":"47-running-appium-from-source-or-the-latest-beta","path":"/editions/47-running-appium-from-source-or-the-latest-beta","news":null,"tags":["bash","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0046.md","num":46,"rawMd":"\n\nIf you're anything like me, sometimes you just want to send arbitrary keystrokes to an app, without necessarily having found a text input field yet. Actually, just kidding; this is not a genuine hobby of mine, but it's a nice way to introduce this edition's topic! It is actually possible to use the new W3C Actions API not only to [send pointer actions](https://appiumpro.com/editions/29), but also to send key input. At the moment, this is supported in Appium's Android drivers.\n\nThe use cases for this feature are varied and probably not too common. Some apps require keyboard input on non-text fields, or have elements that can't be directly accessed (maybe as part of a game that implements its own keyboard, for example).\n\nFor the simple purpose of sending keystrokes, regardless of whether an element is 'focused' or not, the Java client has as nice easy method for doing this, utilizing the `Actions` class:\n\n```java\nActions a = new Actions(driver);\na.sendKeys(\"foo\");\na.perform();\n```\n\nWe construct an instance of `Actions` by passing in our session, register a `sendKeys` action on it with the characters we want to type, and then call `perform()` to make it all happen. The fact that we have to explicitly call `perform()` means we could register a number of key inputs, perhaps with a series of `wait`s in between, or perhaps mixed together with some pointer inputs as well.\n\nThis is all well and good, but what if we want lower-level control over the typing? What if we want to press multiple characters at once, or hold down a meta key (like SHIFT) while typing another character? In that case we'll need to explore the `KeyInput` class. The way that we use it is very similar to the way we use the `PointerInput` class in the [gesture actions guide](https://appiumpro.com/editions/29). First, we define a `Sequence` to contain our actions. Then, we define a `KeyInput` we use to generate the key actions (`KeyDown` or `KeyUp` on specific keys) we will register. Then, we register our actions with the overall sequence, and finally `perform()` that sequence with our driver.\n\nIn the following example, we type \"Foo\" into the app, and get the capital \"F\" by using a combination of keystrokes that overlap in time:\n```java\nKeyInput keyboard = new KeyInput(\"keyboard\");\nSequence sendKeys = new Sequence(keyboard, 0);\n\nsendKeys.addAction(keyboard.createKeyDown(Keys.SHIFT.getCodePoint()));\nsendKeys.addAction(keyboard.createKeyDown(\"f\".codePointAt(0)));\nsendKeys.addAction(keyboard.createKeyUp(\"f\".codePointAt(0)));\nsendKeys.addAction(keyboard.createKeyUp(Keys.SHIFT.getCodePoint()));\n\nsendKeys.addAction(keyboard.createKeyDown(\"o\".codePointAt(0)));\nsendKeys.addAction(keyboard.createKeyUp(\"o\".codePointAt(0)));\n\nsendKeys.addAction(keyboard.createKeyDown(\"o\".codePointAt(0)));\nsendKeys.addAction(keyboard.createKeyUp(\"o\".codePointAt(0)));\n\ndriver.perform(Arrays.asList(sendKeys));\n```\n\nAs you can see, this strategy is quite a bit more verbose, since we have to register both the down and up state of every key we want to add to the sequence. (Of course, in real test code we would probably create some helper method to reduce boilerplate here). It's important to note that these low-level methods use character code points rather than characters themselves. And we're making good use of the built-in `Keys` class from the Selenium client, which lets us access the `SHIFT` key without needing to look up its code point ourselves.\n\nThat's all there is to it! You can get pretty fancy, of course, because you're not limited to the usual ASCII keys and can press multiple keys at once, or in a time sequence of your choosing. Have a look at the full code sample below, where both strategies discussed above are represented:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.Arrays;\nimport org.junit.After;\nimport org.junit.Assert;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.Keys;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.interactions.Actions;\nimport org.openqa.selenium.interactions.KeyInput;\nimport org.openqa.selenium.interactions.Sequence;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition046_W3C_Keys {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.0/TheApp-v1.8.0.apk\";\n private By loginScreen = MobileBy.AccessibilityId(\"Login Screen\");\n private By username = MobileBy.AccessibilityId(\"username\");\n\n private AppiumDriver driver;\n private WebDriverWait wait;\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n\n caps.setCapability(\"app\", APP);\n driver = new AppiumDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testSendKeysAction() {\n wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen)).click();\n WebElement usernameField = driver.findElement(username);\n usernameField.click();\n Actions a = new Actions(driver);\n a.sendKeys(\"foo\");\n a.perform();\n Assert.assertEquals(\"foo\", usernameField.getText());\n }\n\n @Test\n public void testLowLevelKeys() {\n wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen)).click();\n WebElement usernameField = driver.findElement(username);\n usernameField.click();\n\n KeyInput keyboard = new KeyInput(\"keyboard\");\n Sequence sendKeys = new Sequence(keyboard, 0);\n\n sendKeys.addAction(keyboard.createKeyDown(Keys.SHIFT.getCodePoint()));\n sendKeys.addAction(keyboard.createKeyDown(\"f\".codePointAt(0)));\n sendKeys.addAction(keyboard.createKeyUp(\"f\".codePointAt(0)));\n sendKeys.addAction(keyboard.createKeyUp(Keys.SHIFT.getCodePoint()));\n\n sendKeys.addAction(keyboard.createKeyDown(\"o\".codePointAt(0)));\n sendKeys.addAction(keyboard.createKeyUp(\"o\".codePointAt(0)));\n\n sendKeys.addAction(keyboard.createKeyDown(\"o\".codePointAt(0)));\n sendKeys.addAction(keyboard.createKeyUp(\"o\".codePointAt(0)));\n\n driver.perform(Arrays.asList(sendKeys));\n\n Assert.assertEquals(\"Foo\", usernameField.getText());\n }\n\n}\n```\n\n(Don't forget to check out the full code sample inside the runnable project [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition046_W3C_Keys.java))\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Sending Arbitrary Keystrokes With The Actions API","shortDesc":"The W3C Actions API is about more than building complex touch gestures. It extends to full control over the keyboard (virtual or real), and allows for fine-grained control of keystrokes. Using this portion of the Actions API, you can trigger key input in your app without having to first find an element and then use `sendKeys` on it specifically.","liveAt":"2018-12-05 10:00","canonicalRef":"https://www.headspin.io/blog/sending-arbitrary-keystrokes-with-the-actions-api"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/46-sending-arbitrary-keystrokes-with-the-actions-api","slug":"46-sending-arbitrary-keystrokes-with-the-actions-api","path":"/editions/46-sending-arbitrary-keystrokes-with-the-actions-api","news":{"rawMd":"It has come to my attention that the formatting of the code samples is off for many of you. I've reached out to my newsletter service and they've confirmed it's a bug on their end and are working to fix it. Hopefully things will be back to normal soon! In the meantime, you can always read the code samples as part of the online version of the articles at [appiumpro.com](https://appiumpro.com).\n\nAlso, don't forget to [submit a talk proposal](https://confengine.com/appium-conf-2019/proposals) to AppiumConf 2019! We're beginning to review and accept proposals so now is a great time to get feedback on your proposal from the conference committee.\n","num":46,"mdPath":"/vercel/path0/content/news/0046.md"},"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0045.md","num":45,"rawMd":"\n\nIt's very common with modern mobile devices to rely on virtual \"assistants\" to get tasks done, whether in a handsfree situation utilizing voice commands, or just to save the trouble of tapping out search queries. On iOS these interactions take place through the Siri interface, for which developers can create custom integrations using [SiriKit](https://developer.apple.com/sirikit/).\n

\n \"Hey\n

\n\nHow on earth would you test this aspect of your app's behavior? Ideally you'd be able to have a recording of the particular voice command or phrase used to trigger your app's Siri integration, which you could then somehow apply to the simulator or device under test. This is not currently possible, outside of rigging up some speakers!\n\nFortunately, we don't need to go to such lengths, because Appium has recently added a command (as of Appium 1.10), that lets you specify the text you want Siri to parse, _as if_ it had been spoken by a person. This is great, since we don't need to test Siri itself--just its connection to our app.\n\nThe command itself is accessible via the `executeScript` \"mobile\" interface:\n\n```java\nHashMap args = new HashMap<>();\nargs.put(\"text\", \"Hey Siri, what's happening?\");\ndriver.executeScript(\"mobile: siriCommand\", args);\n```\n\nEssentially, we construct an options hash with our desired text string, and pass it to the `siriCommand` \"mobile\" method. We can run this command at any point in our automation, and it will take care of getting to the Siri prompt for us as well (we don't need to long-hold the home button, for example).\n\nAt this point we can use the typical native automation methods to verify Siri's response on the screen, tap on action items, etc...\n\nThat's basically it! There's not much to it. So let's have a look at a full example that asks Siri a math question (What's two plus two?) and verifies the result (notice how the result text shows up as accessibility IDs, which I found by looking at the page source, though I could also have used Appium Desktop):\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.MobileElement;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition045_iOS_Siri {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.8.0/TheApp-v1.8.0.app.zip\";\n\n private IOSDriver driver;\n private WebDriverWait wait;\n\n private By siriCalcQ = MobileBy.AccessibilityId(\"2 + 2 =\");\n private By siriCalcA = MobileBy.AccessibilityId(\"4\");\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"12.1\");\n caps.setCapability(\"deviceName\", \"iPhone 8\");\n caps.setCapability(\"noReset\", true);\n caps.setCapability(\"app\", APP);\n\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 20);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testSiri() {\n HashMap args = new HashMap<>();\n args.put(\"text\", \"What's two plus two?\");\n driver.executeScript(\"mobile: siriCommand\", args);\n wait.until(ExpectedConditions.presenceOfElementLocated(siriCalcQ));\n wait.until(ExpectedConditions.presenceOfElementLocated(siriCalcA));\n }\n}\n```\n\n(Don't forget to check out the full code sample inside the runnable project [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition045_iOS_Siri.java))\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Automating Voice Commands With Siri","shortDesc":"'Siri, how do I automate you with Appium?' While Siri won't have the answer to that question, this edition of Appium Pro does! You can test your app's SiriKit integrations using Appium's `siriCommand` method.","liveAt":"2018-11-28 10:00","canonicalRef":"https://www.headspin.io/blog/automating-voice-commands-with-siri"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/45-automating-voice-commands-with-siri","slug":"45-automating-voice-commands-with-siri","path":"/editions/45-automating-voice-commands-with-siri","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0044.md","num":44,"rawMd":"\n\nToday's web applications are not pieced together using only the standard HTML elements of yesteryear. We now have the ability to use [Web Components](https://developer.mozilla.org/en-US/docs/Web/Web_Components) as a way to encapsulate styling and functionality for particular components, which means we can easily share them and reuse them within and across apps.\n\n### ShadowDOM\n\nThis is great, but when it comes to _testing_ this new breed of web app, things get more complicated. The way Web Components are implemented, the \"interior\" of a component does not appear in the DOM; instead, it appears in a separate tree called a \"Shadow DOM\", which is linked into the \"main\" DOM at the point we want it to appear. This separation is a good thing, because we don't want unrelated style definitions for our main document to be able to target the interior of a 3rd-party component. But what if we want to make use of an element located in the interior of a Web Component? We can't just access it as if it were part of the DOM, because it's hidden away inside its own special Shadow DOM.\n\nLuckily, a JavaScript API is available for getting the root of the Shadow DOM attached to a given element:\n\n`element.shadowRoot`\n\nThe `shadowRoot` of an element is itself an element, namely the root element _inside_ the component. Using the `executeScript` method of our Appium driver, we can actually retrieve this shadow root as a standard `WebElement`:\n\n```java\n// first, find the externally visible element\nWebElement component = driver.findElement(By.tagName(\"custom-component\"));\n\n// now we can use it to find the shadow root\nString getShadow = \"return arguments[0].shadowRoot;\";\nWebElement shadowRoot = (WebElement) driver.executeScript(getShadow, component);\n```\n\nWhat's going on here? Basically we're constructing a little snippet of JS which will be run in the context of the currently-loaded page. But we're not running it free of any context---we're actually passing in an element we've found as an argument to the script. Appium knows how to convert your client-side `WebElement` into an actual HTML element in the context of the executing JS, which means we can access the standard properties available on HTML elements, including `shadowRoot`. Finally, we're simply returning the shadow root and casting it to a `WebElement` so we can work with it in our test script.\n\nNow, if we were dealing with Selenium, we would be able to use this `shadowRoot` element just like any other element (for example in order to find an `input` field inside the component):\n\n```java\nWebElement innerElement = shadowRoot.findElement(By.tagName(\"input\"));\n```\n\nHowever, due to bugs and/or limitations in the technologies that Appium relies on (Chromedriver for Android, and the Selenium Atoms / remote debug protocol for Safari), this will not work. We'll get different errors on the two platforms, but either way we're stuck. What to do?\n\nJavaScript to the rescue again! Let's use the web APIs for searching for elements, instead of relying on the WebDriver `findElement` command:\n\n```java\n// first, find the externally visible element\nWebElement component = driver.findElement(By.tagName(\"custom-component\"));\n\n// now use the 'querySelector' API to directly find the inner element we want\nString getInnerEl = \"return arguments[0].shadowRoot.querySelector('input');\";\nWebElement innerElement = (WebElement) driver.executeScript(getInnerEl, component);\n```\n\nWith this little trick, we're able to now use `innerElement` just as if it were a regular old main DOM element, for example with commands like `innerElement.isSelected()` and so on. And that's how we can work with elements inside the shadow DOM! How did we know we were dealing with a shadow DOM to begin with? It's easy to tell by using the dev tools inspector in a browser like Firefox.\n\n### Closed Shadow DOMs\n\nI've omitted one wrinkle, which is that shadow DOMs come in two flavors: open and closed. Closed shadow DOMs are unfortunately not accessible via the `shadowRoot` property, which means the strategy above will not work in accessing them. Luckily, closed shadow DOMs are considered unnecessary and are not too common. There are ways of dealing with them, nonetheless, which we will cover in another edition of Appium Pro.\n\n### Sorry, Safari\n\nDue to the differences in how Appium supports testing website on Android vs iOS, there is another difference you have to consider in writing your tests, which is that for Safari, using any element found within a shadow DOM will always generate a `StaleElementException` when you try and use it. This means the strategy above (where we get a `WebElement` out of a shadow DOM using `executeScript`), is not going to work.\n\nInstead, for Safari, we have to stay completely within JS-land if we want to successfully automate the shadow DOM. For example, let's say our goal was to check the state of a checkbox input inside of a custom component. With Chrome, we are able to simply find the checkbox using the strategy above, and call `isSelected()` on it to determine whether it's checked. With Safari, calling `isSelected()` will result in a `StaleElementException`. So what we must do instead is something like the following:\n\n```java\n// first, find the externally visible element\nWebElement component = driver.findElement(By.tagName(\"custom-component\"));\n\n// now find and determine checked status of element all using JS\nString getChecked = \"return arguments[0].shadowRoot.querySelector('input').checked;\";\nboolean checked = (boolean) driver.executeScript(getChecked, component);\n// do something with checked value\n```\n\nThis is a completely reliable automation technique, however it relies purely on JS and doesn't stay within the typical WebDriver API usage, which is not ideal.\n\nIt's worth pointing out that if you have to automate both Chrome and Safari, the pure-JS tactic I just described will work for _both_; so in cross-platform situations it's probably better to use this latter technique rather than mixing two different strategies together.\n\nHave a look at a full sample which pierces the shadow DOM of a [Material Web Component](https://material.io/develop/) to determine the input state of a [switch](https://material-components.github.io/material-components-web-catalog/#/component/switch). (Of course the designers of this component were smart and made that checked state bubble up to a property on the component itself, so piercing the shadow DOM is not strictly necessary. It is however useful for instruction purposes here).\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition044_Shadow_DOM {\n\n private static By SWITCH_COMPONENT = By.xpath(\"//mwc-switch[1]\");\n\n private static String SHADOWED_INPUT = \"return arguments[0].shadowRoot.querySelector('input')\";\n private static String INPUT_CHECKED = SHADOWED_INPUT + \".checked\";\n private static String CLICK_INPUT = SHADOWED_INPUT + \".click()\";\n private static String URL = \"https://material-components.github.io/material-components-web-components/demos/switch.html\";\n\n @Test\n public void testShadowDom_iOS() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"12.1\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"browserName\", \"Safari\");\n\n // Open up Safari\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n try {\n testShadowElementsWithJS(driver);\n } finally {\n driver.quit();\n }\n }\n\n @Test\n public void testAppiumProSite_Android() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"browserName\", \"Chrome\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n\n // Open up Chrome\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n try {\n testShadowElementsAsNative(driver);\n testShadowElementsWithJS(driver);\n } finally {\n driver.quit();\n }\n }\n\n public void testShadowElementsAsNative (AppiumDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n driver.get(URL);\n\n // find the web component\n WebElement switchComponent = wait.until(ExpectedConditions.presenceOfElementLocated(\n SWITCH_COMPONENT));\n\n // use it to find the inner control\n WebElement nativeCheckbox = (WebElement) driver.executeScript(SHADOWED_INPUT, switchComponent);\n\n // use the standard API to determine whether the control is checked\n Assert.assertEquals(nativeCheckbox.isSelected(), false);\n\n // use the standard API to change the checked status\n switchComponent.click();\n\n // and finally verify the new checked state after the click\n Assert.assertEquals(nativeCheckbox.isSelected(), true);\n }\n\n public void testShadowElementsWithJS(AppiumDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n driver.get(URL);\n\n // find the web component\n WebElement switchComponent = wait.until(ExpectedConditions.presenceOfElementLocated(\n SWITCH_COMPONENT));\n\n // pierce shadow dom to get checked status of inner control, and assert on it\n boolean checked = (boolean) driver.executeScript(INPUT_CHECKED, switchComponent);\n Assert.assertEquals(false, checked);\n\n // change the state from off to on by clicking inner input\n // (clicking the parent component will not work)\n driver.executeScript(CLICK_INPUT, switchComponent);\n\n // check that state of inner control has changed appropriately\n checked = (boolean) driver.executeScript(INPUT_CHECKED, switchComponent);\n Assert.assertEquals(true, checked);\n }\n}\n```\n\nNotice how both strategies are represented, and that the Android test runs both strategies in the same test, to prove that they both work for Android. Don't forget to also have a look at the sample [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition044_Shadow_DOM.java).\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Working With Web Components (Shadow DOM)","shortDesc":"Web Components are an amazing new web standard that promises to ease the pain of code sharing and UI component reuse across your apps and across the web. They do, however, come with a few headaches for testing, since the very thing that makes Web Components sharable and reusable also makes their inner workings hidden from the outside world. There are several strategies for getting inside the Web Component Shadow DOM, however.","liveAt":"2018-11-21 10:00","canonicalRef":"https://www.headspin.io/blog/working-with-web-components-shadow-dom"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/44-working-with-web-components-shadow-dom","slug":"44-working-with-web-components-shadow-dom","path":"/editions/44-working-with-web-components-shadow-dom","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0042.md","num":42,"rawMd":"\n\nIn the past, we've covered how to simulate [receiving SMS messages](https://appiumpro.com/editions/11) on Android emulators. It is also possible to receive and interact with incoming phone calls! This might be relevant for apps that engage with the phone subsystem in some way, or it might just be a good idea to test that nothing goes wrong in your app when someone takes a call while using it.\n\n(Of course, this only works on emulators; to achieve the same result on a real device, just make sure the real device has a working SIM card and trigger a call to it using one of the available phone call automation services.)\n\nHow can we take advantage of this feature? It's made available in the Appium Java client via the `driver.makeGsmCall()` method. This method takes two parameters:\n\n1. A string representing the phone number which the emulator will show as calling it (this should be purely numbers, no dashes or parentheses).\n2. A member of the `GsmCallActions` enum, namely one of `GsmCallActions.CALL`, `GsmCallActions.CANCEL`, `GsmCallActions.ACCEPT`, or `GsmCallActions.HOLD`. Each of these defines an action the emulator will take with respect to the particular phone number that has called.\n\nBasically, to initiate a call to the emulator, we first make a call like this:\n\n```java\ndriver.makeGsmCall(\"1234567890\", GsmCallActions.CALL);\n```\n\nThis will start the device \"ringing\". At this point it will continue to ring until we follow up with another action. We may, for example, choose to accept the call:\n\n```java\ndriver.makeGsmCall(\"1234567890\", GsmCallActions.ACCEPT);\n```\n\nAt this point the call will be active, and will remain active until we choose to end it:\n\n```java\ndriver.makeGsmCall(\"1234567890\", GsmCallActions.CANCEL);\n```\n\nUsing this series of commands, we can simulate the entire flow of a user receiving a phone call while in our app, accepting the call, continuing to go about her business in the app while the call is active, and then ending the call. At any point in this flow we can continue to run regular Appium commands, which is how we would check that our app continues to function normally even as the call takes place.\n\nThat's it! Take a look at the full example below, which introduces some artificial pauses in the course of the script so you can see the action taking place (otherwise it all happens too quickly):\n\n```java\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.android.GsmCallActions;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition042_Android_Phone {\n\n private AndroidDriver driver;\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.7.1/TheApp-v1.7.1.apk\";\n private String PHONE_NUMBER = \"5551237890\";\n\n @Before\n public void setUp() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", APP);\n\n driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Test\n public void testPhoneCall() throws InterruptedException {\n // do something in our app\n driver.findElementByAccessibilityId(\"Login Screen\").click();\n\n // receive and accept a call\n driver.makeGsmCall(PHONE_NUMBER, GsmCallActions.CALL);\n Thread.sleep(2000); // pause just for effect\n driver.makeGsmCall(PHONE_NUMBER, GsmCallActions.ACCEPT);\n\n // continue to do something in our app\n driver.findElementByAccessibilityId(\"username\").sendKeys(\"hi\");\n Thread.sleep(2000); // pause just for effect\n\n // end the call\n driver.makeGsmCall(PHONE_NUMBER, GsmCallActions.CANCEL);\n Thread.sleep(2000); // pause just for effect\n }\n}\n```\n\nOf course, you can also have a look at the sample [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition042_Android_Phone.java).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Simulating Incoming Phone Calls On Android","shortDesc":"Just as you can generate artificial text messages to an Android emulator, so you can also convince an emulator to believe it's receiving a phone call! Appium has a simple API for triggering incoming phone calls from arbitrary numbers, which can be useful for a variety of reasons, not least of which is making sure your app stays functional during a call.","liveAt":"2018-11-07 10:00","canonicalRef":"https://www.headspin.io/blog/simulating-incoming-phone-calls-on-android"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/42-simulating-incoming-phone-calls-on-android","slug":"42-simulating-incoming-phone-calls-on-android","path":"/editions/42-simulating-incoming-phone-calls-on-android","news":{"rawMd":"As a reminder (the first of many), [AppiumConf 2019](https://appiumconf.com) is a thing! Please do consider [submitting a talk proposal](https://confengine.com/appium-conf-2019/proposals) since we are already reviewing proposals and reaching out to potential speakers.\n","num":42,"mdPath":"/vercel/path0/content/news/0042.md"},"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0039.md","num":39,"rawMd":"\n\nPerhaps the most buzzy of the buzzwords in tech these days is \"AI\" (Artificial Intelligence), or \"AI/ML\" (throwing in Machine Learning). To most of us, these phrases seem like magical fairy dust that promises to make the hard parts of our tech jobs go away. To be sure, AI is (in my opinion) largely over-hyped, or at least its methods and applications are largely misunderstood and therefore assumed to be much more magical than they are.\n\nIt might seem surprising then that this Appium Pro edition is all about how you can use AI with Appium! It's a bit surprising to me too, but by working with the folks at [Test.ai](https://test.ai), the Appium project has developed an AI-powered element finding plugin for use specifically with Appium. What could this possibly be about?\n\nFirst, let's discuss what I mean by \"element finding plugin\". In [a recent addition](https://github.com/appium/appium-base-driver/pull/268) to Appium, we added the ability for third-party developers to create \"plugins\" for Appium that can use an Appium driver together with their own unique capabilities to find elements. As we'll see below, users can access these plugins simply by installing the plugin as an NPM module in their Appium directory, and then using the `customFindModules` capability to register the plugin with the Appium server. (Go ahead and check out the [element finding plugin docs](https://github.com/appium/appium/blob/52e5bf1217f08b963136254222ba2ebef428f0d1/docs/en/advanced-concepts/element-finding-plugins.md) if you want a fuller scoop).\n\nThe first plugin we worked on within this new structure was one that incorporates a machine learning model from Test.ai designed to classify app icons, the training data for which was just [open-sourced](https://www.kaggle.com/testdotai/common-mobile-web-app-icons). This is a model which can tell us, given the input of an icon, what sort of thing the icon represents (for example, a shopping cart button, or a back arrow button). The application we developed with this model was the [Appium Classifier Plugin](https://github.com/testdotai/appium-classifier-plugin), which conforms to the new element finding plugin format.\n\nBasically, we can use this plugin to find icons on the screen based on their appearance, rather than knowing anything about the structure of our app or needing to ask developers for internal identifiers to use as selectors. For the time being the plugin is limited to finding elements by their visual appearance, so it really only works for elements which display a single icon. Luckily, these kinds of elements are pretty common in mobile apps.\n\nThis approach is more flexible than existing locator strategies (like accessibility id, or image) in many cases, because the AI model is trained to recognize icons without needing any context, and without requiring them to match only one precise image style. What this means is that using the plugin to find a \"cart\" icon will work across apps and across platforms, without needing to worry about minor differences.\n\nSo let's take a look at a concrete example, demonstrating the simplest possible use case. If you fire up an iOS simulator you have access to the Photos application, which looks something like this:\n\n

\n \"The\n

\n\nNotice the little magnifying glass icon near the top which, when clicked, opens up a search bar:\n\n

\n \"The\n

\n\nLet's write a test that uses the new plugin to find and click that icon. First, we need to follow the [setup instructions](https://github.com/testdotai/appium-classifier-plugin#testai-classifier-plugin-for-appium) from the plugin's README, to make sure everything will work. Then, we can set up our Desired Capabilities for running a test against the Photos app:\n\n```java\nDesiredCapabilities caps = new DesiredCapabilities();\ncaps.setCapability(\"platformName\", \"iOS\");\ncaps.setCapability(\"platformVersion\", \"11.4\");\ncaps.setCapability(\"deviceName\", \"iPhone 6\");\ncaps.setCapability(\"bundleId\", \"com.apple.mobileslideshow\");\n```\n\nNow we need to add some new capabilities: `customFindModules` (to tell Appium about the AI plugin we want to use), and `shouldUseCompactResponses` (because the plugin itself told us we need to set this capability in its setup instructions):\n\n```java\nHashMap customFindModules = new HashMap<>();\ncustomFindModules.put(\"ai\", \"test-ai-classifier\");\n\ncaps.setCapability(\"customFindModules\", customFindModules);\ncaps.setCapability(\"shouldUseCompactResponses\", false);\n```\n\nYou can see that `customFindModules` is a capability which has some internal structure: in this case \"ai\" is the shortcut name for the plugin that we can use internally in our test, and \"test-ai-classifier\" is the fully-qualified reference that Appium will need to be able to find and require the plugin when we request elements with it.\n\nOnce we've done all this, finding the element is super simple:\n\n```java\ndriver.findElement(MobileBy.custom(\"ai:search\"));\n```\n\nHere we're using a new _custom_ locator strategy so that Appium knows we want a plugin, not one of its supported locator strategies. Then, we're prefixing our selector with `ai:` to let Appium know which plugin specifically we want to use for this request (because there could be multiple). Of course since we are in fact only using one plugin for this test, we could do away with the prefix (and for good measure we could use the different find command style, too):\n\n```java\ndriver.findElementByCustom(\"search\");\n```\n\nAnd that's it! As I mentioned above, this technology has some significant limitations at the current time, for example that it can really only reliably find elements which are one of the icons that the model has been trained to detect. On top of that, the process is fairly slow, both in the plugin code (since it has to retrieve every element on screen in order to send information into the model), and in the model itself. All of these areas will see improvement in the future, however. And even if this particular plugin isn't useful for your day-to-day, it demonstrates that concrete applications of AI in the testing space are not only possible, but actual!\n\nAs always, you can check out a full (hopefully) [working example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition039_AI_For_Appium.java) at Appium Pro's GitHub repo. Happy (locator-angst-free) testing!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Early-Stage AI for Appium Test Automation","shortDesc":"There's been a lot of discussion around the potential ramifications of AI for automated testing. While the debate will no doubt rage on, Appium and Test.ai have teamed up to release a simple integration which allows you to find elements using a machine learning model, making it possible to find icons in your app without knowing anything about your app or looking up selectors.","liveAt":"2018-10-18 12:00","canonicalRef":"https://www.headspin.io/blog/early-stage-ai-for-appium-test-automation"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/39-early-stage-ai-for-appium-test-automation","slug":"39-early-stage-ai-for-appium-test-automation","path":"/editions/39-early-stage-ai-for-appium-test-automation","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0038.md","num":38,"rawMd":"\n\nWe've already seen how to [capture browser logs for iOS](https://appiumpro.com/editions/37); the same can be done for Android! The technique is largely similar but varies in some respects. As a reminder of why we might want to capture logs and errors from a browser, we could:\n\n1. Get under-the-hood information from our app that modulates the way our test works.\n2. Store the logs for future reference; we could give them to an app dev if a test fails, and they might be able to use the logs to pinpoint the nature of an otherwise confusing bug.\n\nAnd of course, just as in the iOS world, we're not limited to capturing console logs; we could make sure any JavaScript errors that are thrown get attached to the console as well, so we track those too (using a little hack like `window.onerror = console.error.bind(console)` or similar).\n\nBut let's skip to the action. With iOS there was some complexity where we used a non-standard `safariConsole` log type, and had to parse some JSON to get the actual data, with Android we can take advantage of the fact that Chrome automation happens via Chromedriver, which supports the full WebDriver protocol all on its own. Given that, we can simply make a request for the `browser` log type:\n\n```java\ndriver.manage().logs().get(\"browser\");\n```\n\nThis returns a list of `LogEntry` objects, and for each one we can retrieve its message (and, say, print it to the terminal):\n\n```java\nfor (LogEntry entry : driver.manage().logs().get(\"browser\")) {\n System.out.println(entry.getMessage());\n}\n```\n\nThat's all there is to it, in terms of the test code itself. However, we do need to use a special `loggingPrefs` capability to convince Chromedriver to turn on capturing of the browser logs for us. In it simplest form, the `loggingPrefs` capability is an object specifying log types and log levels. For our purposes it ends up looking like:\n\n```json\n{\"loggingPrefs\": {\"browser\": \"ALL\"}}\n```\n\nBut in Java-client-world, we can build up this little structure using classes and constants instead! That way our IDE can guide us to the correct structure without having to remember strings like \"loggingPrefs\":\n\n```java\nLoggingPreferences logPrefs = new LoggingPreferences();\nlogPrefs.enable(LogType.BROWSER, Level.ALL);\ncapabilities.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);\n```\n\nWith this capability set, `browser` log retrieval will work, regardless of whether you're automating Chrome itself or a hybrid app. The code sample for this edition is basically the same as for the iOS equivalent, so I added a hybrid test just to show that it works equally well with hybrid apps (though of course we have to switch into the web context before making the call to get the browser logs). Here's the full example (which you can also view [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition038_Android_Web_Console.java)):\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.AndroidDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.ArrayList;\nimport java.util.logging.Level;\nimport javax.annotation.Nullable;\nimport org.junit.After;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.logging.LogEntry;\nimport org.openqa.selenium.logging.LogType;\nimport org.openqa.selenium.logging.LoggingPreferences;\nimport org.openqa.selenium.remote.CapabilityType;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition038_Android_Web_Console {\n\n private AndroidDriver driver;\n private String APP_ANDROID = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.7.1/TheApp-v1.7.1.apk\";\n private static By hybridScreen = MobileBy.AccessibilityId(\"Webview Demo\");\n private static By urlInput = MobileBy.AccessibilityId(\"urlInput\");\n\n @After\n public void tearDown() {\n if (driver != null) {\n driver.quit();\n }\n }\n\n @Nullable\n private String getWebContext(AppiumDriver driver) {\n ArrayList contexts = new ArrayList(driver.getContextHandles());\n for (String context : contexts) {\n if (!context.equals(\"NATIVE_APP\")) {\n return context;\n }\n }\n return null;\n }\n\n @Test\n public void testLogging_Chrome() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"browserName\", \"Chrome\");\n LoggingPreferences logPrefs = new LoggingPreferences();\n logPrefs.enable(LogType.BROWSER, Level.ALL);\n capabilities.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);\n\n driver = new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n loggingRoutine(driver);\n }\n\n @Test\n public void testLogging_Hybrid() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"app\", APP_ANDROID);\n LoggingPreferences logPrefs = new LoggingPreferences();\n logPrefs.enable(LogType.BROWSER, Level.ALL);\n capabilities.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);\n\n driver = new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n // get to webview screen and enter webview mode\n wait.until(ExpectedConditions.presenceOfElementLocated(hybridScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(urlInput));\n driver.context(getWebContext(driver));\n\n // now we can run the same routine as for the browser\n loggingRoutine(driver);\n }\n\n public void loggingRoutine(AndroidDriver driver) {\n driver.get(\"https://appiumpro.com/test\");\n driver.executeScript(\"window.onerror=console.error.bind(console)\");\n driver.executeScript(\"console.log('foo.');\");\n driver.executeScript(\"console.warn('bar?');\");\n driver.findElementById(\"jsErrLink\").click();\n\n for (LogEntry entry : driver.manage().logs().get(\"browser\")) {\n System.out.println(entry.getMessage());\n }\n }\n\n}\n```\n\n(NB: you'll need to make sure you're using Appium 1.9.2-beta or greater, since before this, the `loggingPrefs` capability was not passed correctly to Chromedriver).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Capturing Browser Errors and Logs in Android Web/Hybrid Apps","shortDesc":"When using Chrome on Android, or Chrome-backed webviews, it is possible to retrieve logs written by web applications (or hybrid applications) to the browser console. Retrieving these logs is useful for a number of reasons, from observing app state to saving app-internal logs along with test artifacts.","liveAt":"2018-10-10 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-browser-errors-and-logs-in-android-web-hybrid-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/38-capturing-browser-errors-and-logs-in-android-webhybrid-apps","slug":"38-capturing-browser-errors-and-logs-in-android-webhybrid-apps","path":"/editions/38-capturing-browser-errors-and-logs-in-android-webhybrid-apps","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0037.md","num":37,"rawMd":"\n\nWe've talked a few times about automating [web](https://appiumpro.com/editions/4) and [hybrid](https://appiumpro.com/editions/17) apps with Appium. One feature of such apps is the ability to write information to the browser console. Whether or not this console is visible to the end user is irrelevant---app developers often announce interesting information about the state of an app in the form of browser console messages. It would be useful to be able to retrieve this information as part of our test, for two main reasons:\n\n1. To make assertions on app state (for example if we're waiting for a 3rd-party JS library to load, and the only way we have of determining its state is by inspecting the console)\n2. To save the record of an app's behavior during a test run for future inspection (for example to assist in debugging failed tests, or in reporting bugs caught by a test).\n\nSomething closely related to the second purpose is the need to capture JS _errors_; it's extremely useful to detect whether the web/hybrid app has thrown any errors during the course of a test! Using a simple little hack, it's actually quite easy to get any JS error logged to the browser console. We simply have to run this little bit of JS in the page:\n\n```js\nwindow.onerror = console.error.bind(console);\n```\n\nThis basically tells the browser to direct information about any error it detects on the page to the console.\n\nOf course, all of this is moot if there's no way to use Appium to retrieve this information. Happily, there is! Let's take a look at how to retrieve the browser logs for Safari (or any other iOS hybrid app).\n\nWe begin by using the log retrieval mechanism built into the Appium client (by way of the Selenium client included within), and specifying a specific, custom _log type_ `safariConsole`:\n\n```java\ndriver.manage().logs().get(\"safariConsole\");\n```\n\nIn Java, this will return a `LogEntries` object which contains a set of (you guessed it) log entries: one per console message which has been received since the last time we retrieved the entries. Each entry has certain data associated with it: a _message_, a _level_ and a _timestamp_. In the case of `safariConsole` logs, the message is itself a JSON string, which contains lots of information (the console message itself, a stack trace, and so on). This means that we'll typically need to convert the message from its JSON string format into an object we can work with in our code. Let's say we wanted to print out the numeric timestamp, the severity level, and the message of everything that's been logged to the console. We can do so using the following code:\n\n```java\nfor (LogEntry entry : driver.manage().logs().get(\"safariConsole\")) {\n HashMap consoleEntry = new Json().toType(entry.getMessage(), Json.MAP_TYPE);\n System.out.println(String.format(\n \"%s [%s] %s\",\n entry.getTimestamp(),\n entry.getLevel(),\n consoleEntry.get(\"text\")));\n}\n```\n\nEssentially, what we're doing is going through each entry in the list of entries, turning the message into a key/value pair so we can extract specific data from it, and using some of the general entry API methods as well (to get the time and severity level of the message). Of course, this is just one example of how we could use the `driver.manage().logs()` feature. We could do all kinds of things with it. But when would we want to do something like above? It would sit very naturally in some kind of teardown method. Instead of writing to stdout, we could write the console messages to a file and save it along with other test artifacts in case we need it later on to figure out why a test failed.\n\nWe could also use the `window.onerror` hack I showed above, to make sure that any JS errors are brought to our attention; we could even be very strict with our developers and automatically fail a test if any `SEVERE`-level log entries come through! We don't even need our developers' permission to set up this little error handler; we can do it ourselves using `executeScript` at the beginning of our test:\n\n```java\ndriver.executeScript(\"window.onerror=console.error.bind(console);\");\n```\n\n(2 caveats here with this simplistic hack: first, we'll need to run this little script _after_ the page has loaded, which means we won't catch errors that happen on page load. Second, we'd overwrite any other handler attached to the `onerror` event, which might break the intended behavior of the app if such a handler was already in use).\n\nAt any rate, using the `safariConsole` log type, we're able to get all kinds of interesting information from the browser console, whether in Safari or even hybrid apps. Have a look at the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition037_iOS_Web_Console.java) to see how I've put everything together in the context of a real-world example:\n\n```java\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.json.Json;\nimport org.openqa.selenium.logging.LogEntry;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition037_iOS_Web_Console {\n private IOSDriver driver;\n\n @Before\n public void setUp() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"browserName\", \"Safari\");\n\n driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @After\n public void tearDown() {\n if (driver != null) {\n for (LogEntry entry : driver.manage().logs().get(\"safariConsole\")) {\n HashMap consoleEntry = new Json().toType(entry.getMessage(), Json.MAP_TYPE);\n System.out.println(String.format(\n \"%s [%s] %s\",\n entry.getTimestamp(),\n entry.getLevel(),\n consoleEntry.get(\"text\")));\n }\n driver.quit();\n }\n }\n\n @Test\n public void testLogging() {\n driver.get(\"https://appiumpro.com/test\");\n driver.executeScript(\"window.onerror=console.error.bind(console)\");\n driver.executeScript(\"console.log('foo.');\");\n driver.executeScript(\"console.warn('bar?');\");\n driver.findElementById(\"jsErrLink\").click();\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Capturing Browser Errors and Logs in iOS Web/Hybrid Apps","shortDesc":"A lot of information is transmitted through the browser console, some of which is very useful, especially in cases of app errors. Using the log retrieval capabilities of the Appium client, we can gather browser console messages for storage as a test artifact or even to make assertions on app state.","liveAt":"2018-10-03 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-browser-errors-and-logs-in-ios-web-hybrid-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/37-capturing-browser-errors-and-logs-in-ios-webhybrid-apps","slug":"37-capturing-browser-errors-and-logs-in-ios-webhybrid-apps","path":"/editions/37-capturing-browser-errors-and-logs-in-ios-webhybrid-apps","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0036.md","num":36,"rawMd":"\n\nIt's easy to forget that Appium is used not only for native apps but also for testing websites by automating mobile browsers like iOS's Mobile Safari. And Appium's support of the WebDriver API for Safari is so extensive that it's easy to forget it's not the exact same thing as Selenium!\n\nActually, to facilitate automation of Safari, Appium _does_ make use of some of the Selenium Project's code (the [Selenium Atoms](https://github.com/SeleniumHQ/selenium/tree/master/javascript/webdriver/atoms)) -- thanks Selenium friends! The Atoms are a way to get the Selenium/WebDriver API working via JavaScript, rather than native browser code, which is a great thing for Appium since the only way we can automate Mobile Safari is via JavaScript (over a remote debugging connection).\n\nUsually, this little implementation detail is minor enough to ignore. Sometimes, however, it can show up in certain limitations. At the end of the day, Atoms-driven actions are just JavaScript, meaning that clicks and taps are simply simulated by the firing of JavaScript events (rather than registering with the native iOS touch interfaces, and trickling down into Safari from the \"top\", as it were).\n\nOne clue that this is happening to you is when you encounter the following scenario:\n\n1. You find an element (no element not found error)\n2. You click the element (no error thrown on click)\n3. Nothing happens in your app\n\nThis is odd! The element was found, and Appium thinks the click was successful. So what could have gone wrong? Well, the answers can be various and complex. One common culprit is that app developers often assign app behaviors to non-standard JavaScript events. For example, if I decide that (in the mobile version of my web app) a certain action should only take place after the `touchend` event is fired on a particular element, then I will not be able to automate tapping that element via the Selenium Atoms.\n\nIf I load up the website in an iOS simulator or device, I'll be able to trigger the action just fine. This is because Safari knows to trigger that particular event when it detects a \"real\" touch (when the iOS subsystems notify Safari that a tap has happened). But all the Atoms do is fire one or more JavaScript events at the element under consideration---and in this case, it won't do the trick.\n\nTo work around this and other related issues, Appium has a special capability: `nativeWebTap`. When it is set to true, Appium will perform some magic behind the scenes any time you call `click()` on an element found in the browser. It will essentially try to determine the location of that element on the webpage, translate that location to native screen coordinates, account for any system bars and the like, and then generate a _native_ (XCUITest-driven) tap on that location. If all goes well, the result is that the iOS subsystems detect a touch at a certain place, pass that info on to Safari, which then determines that an element has been tapped.\n\nThe net result is that you should be back in business: you'll be able to trigger your app's action and continue testing the user flow. You might also feel a little sense of satisfaction that you generated a more \"real\" tap! (Or not--what's really real when it comes to testing?)\n\nBecause Appium has to do some mathematicky stuff to make `nativeWebTap` work, it's best not to use the capability unless you have some reason to. And an alternative approach might be to simply bug your app developer and ask them if they used some non-standard touch handler, and whether they could just stop and do things the normal way instead. Another option would be to simply fight fire with fire (pun very much intended): figure out what event the developer decided the element is listening for, and fire that event on the element yourself using `executeScript`! It's up to you--either way, Appium's got you covered.\n\nBy way of example, I put together a little [test website](https://appiumpro.com/test) that demonstrates the situation I've been describing. Clicking the link in a desktop browser will do nothing (as intended), but clicking it in Mobile Safari will work just fine. Trying to make the click succeed without using `nativeWebTap` will be impossible, since the navigation behavior is listening for a different JavaScript event.\n\nIf you're interested in seeing this discrepancy yourself (with and without `nativeWebTap`), have a look at the [full code example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition036_NativeWebTap.java) for this edition. (I don't reproduce it here since the only item of interest is the use of the capability itself).\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Using The 'nativeWebTap' Capability","shortDesc":"When automating web applications using Appium, we sometimes run into situations where clicking an element doesn't appear to do anything in our tests, even though clicking the element manually works. The nativeWebTap capability comes to our rescue!","liveAt":"2018-09-26 10:00","canonicalRef":"https://www.headspin.io/blog/using-the-nativewebtap-capability"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/36-using-the-nativewebtap-capability","slug":"36-using-the-nativewebtap-capability","path":"/editions/36-using-the-nativewebtap-capability","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0035.md","num":35,"rawMd":"\n\nI've written a couple editions of Appium Pro on the topic of [finding elements reliably](https://appiumpro.com/editions/20), including why you should [consider not using XPath](https://appiumpro.com/editions/8) at all as part of your element-finding strategy. There are two main reasons for not using XPath:\n\n1. XPath queries can be strictly hierarchical in nature, with the result that any change in the structure of your app (even accidental, or OS-caused) means a failure to find an element (or worse, finding the wrong element).\n2. With Appium specifically, using the XPath locator strategy can be expensive and slow, because of the extra work required to convert an app's UI hierarchy to XML, and then to match up found XML nodes with actual UI elements.\n\nThese are really good reasons to avoid XPath if at all possible. Sometimes, however, there is no alternative! (This goes for both Appium and Selenium, by the way: everything I'm about to say is equally valid for both automation tools.) Sometimes, you've been able to determine that XPath is, in your particular case, not actually expensive or slow (how did you determine this? You tried it!), and so you might prefer XPath to some of the platform-specific locator strategies (iOS Predicate String, or Android UISelector) in order to help your test code be more cross-platform.\n\nEither way, what's important is that you write _good_ XPath queries. In essence, these are queries which are _anchored by elements uniquely identified by unchanging criteria_. Let's first take a look at some examples of queries which fail to meet this description:\n\n* `//*` (this is the worst possible query. It selects every single element!)\n* `//android.widget.Layout/android.widget.Layout[3]/android.widget.Layout/android.widget.TextView[2]` (this query relies too much on hierarchical structure. If an extra layout is added to the hierarchy, or anything is shuffled around, it is likely to break. In addition, the query is not cross-platform).\n\nHow can we make better queries? The first is to remember that XPath offers the use of _predicates_, which allow the restriction of matched nodes based on special criteria. For example, we can find any element which has a certain attribute (say a `text` attribute):\n\n* `//*[@text=\"foo\"]`\n\n(How did we know that the `text` attribute was a thing? We looked at the XML source of the app of course! Maybe by using [Appium Desktop](https://github.com/appium/appium-desktop).) We can also use XPath _functions_ for predicates involving functions other than equality:\n\n* `//*[contains(@text, \"f\")]` (find any element which contains the letter \"f\").\n\nIn practice, the same attributes are not in use within XML sources produced for iOS and Android, so using predicates is often not a cross-platform approach. It can become cross-platform, however, when you remember that XPath also allows _boolean operations_!\n\n* `//*[@text=\"foo\" or @label=\"foo\"]` (find any element whose `text` (Android) or `label` (iOS) attribute is \"foo\").\n\nNotice that I haven't really been referring to the specific type of node in these searches, and instead I have been using the wildcard matcher (`*`). Doesn't this severely hurt performance since the XPath engine has to search so many more nodes? Not really. XPath searches are pretty fast, unless you have a truly gigantic hierarchy, and that doesn't happen too often. (Of course you can always optimize for a given platform by including the type of element you are expecting). For Appium, the truly expensive part of using XPath is in generating the XML, and matching found XPath nodes to native UI elements---not in the XPath search itself.\n\nFor our last consideration, consider an example where there is really no uniquely identifying information on a particular element. We might have a list view with many different elements inside, of an indeterminate ordering, many of which have duplicate text. We're interested in just one of these elements based on external criteria (maybe the text of an ancestor element). Fear not! We can stay in the mostly safe zone of XPath by ensuring that we use an anchor element which _does_ have some unique attribute:\n\n* `//*[@content-desc=\"foo\" or @name=\"foo\"]//*[@content-desc=\"bar\" or @name=\"bar\"]` (a cross-platform accessibility label search for elements called `bar` that descend from elements called `foo`).\n\nIn the above case, we considered the `foo` element our \"anchor\" element, and essentially scoped our query inside of it, making it much more robust. It can work the other way as well, using a child element (or a sibling element) as an anchor:\n\n* `//ancestor::*[*[@text=\"foo\"]][@text=\"bar\"]` (find the ancestor of a node with text \"foo\"; the ancestor must also have text \"bar\").\n\nThese XPath queries may be complex or ugly, but they are not particularly bad from the perspective of maintainability, assuming your app structure guarantees a minimum ancestor/descendant relationship between elements. It simply becomes a process of triangulating an unchanging query for your element based on attributes of other elements! All this to say, if you're not an XPath expert already it's worth finding some good tutorials and making sure you understand the possibilities of XPath for writing better queries, before rejecting the strategy out of hand.\n\nThe way XPath is actually used in practice in many functional testsuites should absolutely be avoided. But this avoidance should not be cargo-culted any more than the use of XPath. In many cases the question of speed and performance with respect to XPath must be determined experimentally, in the context of your particular app. Don't assume without checking that it will be too slow to be useful! Hopefully these reflections and examples help to bring a bit of nuance to the \"XPath is bad\" conversation that I often find myself in, as I'm sure you do as well.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Writing XPath Queries That Work","shortDesc":"We hear a lot (from Appium Pro and other places) about how it's not a good idea to use XPath. That's generally correct. There are times, however, when it is totally fine to use XPath, or when XPath is the only option. In this edition we take a look at how to write good XPath queries to minimize the XPath blast radius.","liveAt":"2018-09-19 10:00","canonicalRef":"https://www.headspin.io/blog/writing-xpath-queries-that-work"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/35-writing-xpath-queries-that-work","slug":"35-writing-xpath-queries-that-work","path":"/editions/35-writing-xpath-queries-that-work","news":{"rawMd":"I am honored that two recent editions of Appium Pro (32 and 33, on finding elements by image) have been translated to Japanese. While I do not read Japanese myself, it appears that Naruhiko Ogasawara has done an amazing job in porting these over, and publishing them on the Selenium Japan site:\n\n* [Appium Pro #32 in Japanese](http://www.selenium.jp/translation/huaxiangniyoruyaosujiansuopart1)\n* [Appium Pro #33 in Japanese](http://www.selenium.jp/translation/appiumhuaxiangniyoruyaosujiansuopart2)\n\nThanks, Naruhiko!\n","num":35,"mdPath":"/vercel/path0/content/news/0035.md"},"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0034.md","num":34,"rawMd":"\n\nWhen we use our phones we're not always just interacting with UI elements inside an app. Mobile devices, particularly Android devices, have an array of physical buttons and keys that often play a role in modulating device state or navigating through apps. It's important to test the use of these keys in your app, but sometimes it's just plain convenient to be able to use them.\n\nLuckily, Appium gives you direct access to these special functions via the Android [KeyEvent](https://developer.android.com/reference/android/view/KeyEvent) library, made available to Appium users with the `pressKey` methods. What are `KeyEvent`s? They're Android system events triggered when users press various keys or buttons, and come in three parts:\n\n1. The keycode of the pressed key\n2. Any meta state information (meta states correspond to meta keys being pressed, such as ALT or CTRL)\n3. Any flags (flags are set by the system, such as whether the key event originated from the soft keyboard)\n\nAppium enables us to actually _generate_ KeyEvents, making the system (and thus any apps under test) believe that a user actually triggered the events. We can construct KeyEvents with each of the three components above (keycode, metastate, and flag). In most cases, we don't need to worry about the metastate or flag, but of course this will depend on what your app is listening for.\n\nIn the simplest case, we can simply call the `pressKey` method (which exists only on `AndroidDriver` of course, since KeyEvents are specific to Android:\n\n```java\ndriver.pressKey(keyEvent);\n```\n\nBut how do we construct the `keyEvent` parameter? It must be of type `KeyEvent`, and at a minimum we have to provide a keycode, which is an integer (represented as the `AndroidKey` enum for the Java client). So let's say we wanted to simulate a click of the hardware \"Back\" button on some Android devices. We can simply write:\n\n```java\ndriver.pressKey(new KeyEvent(AndroidKey.BACK));\n```\n\nAnd the device will behave as though we've triggered the \"Back\" button! Though it wouldn't make sense in this case, we could also have added a metastate to the KeyEvent:\n\n```java\nnew KeyEvent(AndroidKey.BACK)\n .withMetaModifier(KeyEventMetaModifier.SHIFT_ON);\n```\n\nWhat if we want to power off the device? We'll need more than `pressKey` since it's required to hold the power button for a long time. We could instead try the appropriately-named `longPressKey` method:\n\n```java\ndriver.longPressKey(new KeyEvent(AndroidKey.POWER));\n```\n\nAnd that's all there is to it! The real magic lies in seeing everything you have access to using the KeyEvent interface. To see the full list of possible keycodes, simply visit the [Android KeyEvent docs](https://developer.android.com/reference/android/view/KeyEvent) and scroll down through the huge list of constants. You've got everything from alphanumeric keys to media controls. I'm sure some of you will find some interesting uses for this API that the rest of us never considered! Be sure to let me know if you do.\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Simulating Hardware Keys And Key Events On Android","shortDesc":"Appium automation doesn't necessarily stop at the edge of the screen. Your Android device has hardware buttons (power, volume control, etc...), and Appium gives you the ability to automate these as well, using a generic KeyEvent interface from Android.","liveAt":"2018-09-12 10:00","canonicalRef":"https://www.headspin.io/blog/simulating-hardware-keys-and-key-events-on-android"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/34-simulating-hardware-keys-and-key-events-on-android","slug":"34-simulating-hardware-keys-and-key-events-on-android","path":"/editions/34-simulating-hardware-keys-and-key-events-on-android","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0033.md","num":33,"rawMd":"\n\n> This is the second in a two-part series on interacting with screen regions that match a certain image in Appium. If you haven't read [Part 1](https://appiumpro.com/editions/32) yet, do that first!\n\nIn this edition, we'll take a look at some of the advanced usage techniques that go along with Appium's \"find by image\" feature, made available as the `-image` locator strategy. Why do we need any \"advanced\" techniques? The trouble is that image recognition is a bit complicated, and lots of things can get in the way of a successful image match.\n\nFor example, you could send in a reference image that's scaled larger than the size of the device screenshot. This would cause the image match algorithm (implemented in the OpenCV library) to blow up (because how can you look for a reference image inside a screenshot if it's bigger than the screenshot?). Or what if the element you found via image has changed position in between the time you found it and the time you initiated a `tap` command on it? You'd end up tapping a screen coordinate which no longer forms part of your element!\n\nIn each of these cases, Appium can run some logic to help you out (for example, downscaling reference images for you, or automatically re-finding an image element when you attempt to tap it, so that its position is updated if necessary). Appium could just do this on its own, delivering the most robust and reliable image finding experience without you being any the wiser about its magic. The problem is that each of these \"fixes\" requires Appium to spend potentially a considerable amount of time working or talking to the automation engine. In the interests of speed, only basic image element finding functionality is turned on by default. So let's look at the options available to us if, for whatever reason, that basic functionality is not enough.\n\nEach of these options are made available through the Appium [Settings API](https://appium.io/docs/en/commands/session/settings/update-settings/). This is an API only available in Appium (not Selenium) which enables the toggling or resetting of capabilities mid-test. It's used for the same type of parameters as Desired Capabilities, except it allows the Appium client to update the settings at any time and as many times as you like. In the Java client, the Settings API is hidden behind the `HasSettings` interface. If we have an `AndroidDriver` or `IOSDriver` object, it will implement this interface and expose the `setSetting` method. (If we have an `AppiumDriver`, on the other hand, we'll need to typecast to `HasSettings` first).\n\nUsage is extremely simple:\n\n```java\ndriver.setSetting(setting, value);\n```\n\nBasically, we provide a setting name (actually, an element of the `Setting` enum), and a value. Let's take a look at how we can use this to module image element finding.\n\n### Change the image match threshold\n\nWith OpenCV, image matching is not a binary outcome. Instead, there are degrees of match, on the scale between 0 and 1. The scale itself is arbitrary, but 1 represents a pixel-for-pixel perfect match, and 0 represents no comparability whatsoever. With a little bit of experimentation, Appium's default match threshold has been set to `0.4`. This means that, by default, any match attempt which results in a similarity measure of less than `0.4` will be rejected.\n\nFor whatever reason, you might wind up in a situation where `0.4` is too stringent (not letting you find your element), or not stringent enough (giving you matches even when your element is not present). You can adjust it using the `IMAGE_MATCH_THRESHOLD` setting:\n\n```java\ndriver.setSetting(Setting.IMAGE_MATCH_THRESHOLD, 0.2);\n```\n\nHow do you know what value to put? You'll have to experiment, because the scale itself is arbitrary (and non-linear besides). Luckily you can change the threshold for different image element find operations using the Settings API as shown above.\n\n### Change the image element tap strategy\n\nOnce you've found an image element, and call `element.click()` on it, how does Appium know how to tap the element? Without magic, unfortunately. Appium simply takes the bounds of the matched screen region and performs a tap on the center coordinate of those bounds. Of course, Appium has a couple different ways to tap at a point, for example using the W3C Actions API or the older Touch Actions API (see the beginning of [this Appium Pro edition](https://appiumpro.com/editions/29) for an explanation of the history behind these two APIs).\n\nIf Appium's default tap strategy (using W3C Actions) does not work for you (say because you're using a driver which has not been updated to support the W3C Actions API), you can always fall back to the older API, using the `IMAGE_ELEMENT_TAP_STRATEGY` setting:\n\n```java\ndriver.setSetting(Setting.IMAGE_ELEMENT_TAP_STRATEGY, \"touchActions\");\n```\n\n(The two valid options are `\"w3cActions\"` and `\"touchActions\"`)\n\n### Fix screenshot and device size mismatches\n\nAppium's image match algorithms operate on two images: a base image (the one we will attempt to locate the element within), and a reference image or template (the one corresponding to the element we are trying to find). You don't have to worry about the base image: Appium just uses a screenshot from your device, since it represents what's happening at the moment on the screen. But what if the base image (screenshot) and the screen itself don't have the same dimensions? Then the match coordinates returned by the match algorithm would correspond to the screenshot, but not the device. Unfortunately the device is where the eventual tap happens, which means the tap would be happening somewhere other than intended.\n\nWhy would a screenshot not match the dimensions of the screen itself? For a variety of reasons, not least because of the way pixel scaling is handled across platforms. iOS, for example, will say that a screen is 375 pixels wide (\"logical\" width), then happily generate a screenshot which is 750 pixels wide (\"Retina\" width)!\n\nGetting these dimensions matched correctly is so important that Appium does it by default. If, however, you'd prefer not to have Appium spend the CPU cycles making this happen, you can opt out:\n\n```java\ndriver.setSetting(Setting.FIX_IMAGE_FIND_SCREENSHOT_DIMENSIONS, false);\n```\n\n### Fix the reference image size\n\nAs mentioned above, it's necessary for the reference image (template) to have a size smaller than the screenshot for the algorithm to work. Sometimes, when we're generating our reference images (say by snapping a manual screenshot of an element), we're not entirely sure of the dimensions of our template. It could be that the way we captured the template has resulted in its having dimensions larger than what Appium has determined the screenshot dimensions to be.\n\nIf you want to be sure that you can use your template image, no matter what size it happens to be, let Appium know it's OK to resize it for you with this setting:\n\n```java\ndriver.setSetting(Setting.FIX_IMAGE_TEMPLATE_SIZE, true);\n```\n\nBy default, Appium doesn't do this for you because it could hide valuable feedback about potentially incorrect templates (not to mention that it takes some computational time and energy to do the resizing).\n\n### Check for image element staleness\n\nSomething you might be familiar with from the Selenium world is the concept of a \"stale element\". This is an element which, in the time between its being found and its being interacted with, has somehow disappeared. In this case it's not possible to complete the desired interaction (tap, send keys, etc...), and so a `StaleElementException` is thrown.\n\nThe same kind of situation can happen with image elements. What if the element (represented by a set of coordinates returned by the image match algorithm) has gone away in between finding it and tapping on it? Because this would result in Appium tapping on coordinates that no longer represent your element, it attempts to verify the match again whenever a tap is requested. If the match is once again successful, and returns the same coordinates as before, then the tap will take place. If the match is not successful at all, or the coordinates differ, then a `StaleElementException` will be returned instead.\n\nIf you'd prefer to get a bit of a performance improvement and do away with this safety check, you can always turn it off:\n\n```java\ndriver.setSetting(Setting.CHECK_IMAGE_ELEMENT_STALENESS, false);\n```\n\n### Refresh elements automatically\n\nBy default, Appium will let you know with an exception if an image element is stale. But what if the element has not disappeared, only changed position? Maybe it's OK to simply tap on it at its new location without further ado. If you'd prefer to have Appium automatically determine a new match location for image elements when you request a tap, you can use the `UPDATE_IMAGE_ELEMENT_POSITION` setting:\n\n```java\ndriver.setSetting(Setting.UPDATE_IMAGE_ELEMENT_POSITION, true);\n```\n\nThis is set to `false` normally, to preserve the normal situation where you're notified if something has changed in between a find and a tap.\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Finding Elements By Image, Part 2","shortDesc":"We've already seen how to find elements by image, but this doesn't always work out of the box. There are a number of knobs, dials, and switches you can play with (in the form of Appium settings) that help modulate the behavior of the image matching procedures. These will help fine-tune and stabilize your find-by-element usage.","liveAt":"2018-09-05 10:00","canonicalRef":"https://www.headspin.io/blog/finding-elements-by-image-part-2"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/33-finding-elements-by-image-part-2","slug":"33-finding-elements-by-image-part-2","path":"/editions/33-finding-elements-by-image-part-2","news":{"rawMd":"Everybody please welcome Appium Pro's newest sponsor, [HeadSpin](https://headspin.io)! HeadSpin is an Appium-based real device cloud that offers some really interesting features, especially around real-world performance and user experience testing. They're a company that really believes in Appium's potential and is helping make that a reality.\n\nAs it happens, they have also just announced a full-time job opportunity that might be of interest to some of you Appium Pros: it's an engineering position that involves open-source work on Appium itself, as well as a bunch of exciting integration projects. You'd be working closely with the core engineering team to build world-class automation support. Check out the [job description](https://jobs.lever.co/headspin/f09ef2d7-3510-44c3-9ac8-e135324dfc04) and e-mail [brien@headspin.io](mailto:brien@headspin.io) directly if you want to apply. Oh, and don't forget to mention I sent you.\n\nI'm super excited to see more companies like HeadSpin jumping on board with the Appium vision and helping to make it better for all of us!\n","num":33,"mdPath":"/vercel/path0/content/news/0033.md"},"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0032.md","num":32,"rawMd":"\n\nOne of the unfortunate realities of mobile automation is that not every UI element is automatable in practice! This could be because the element is built from a custom class with no accessibility or automation support. It could be because there's no uniquely identifying locator strategy and selector that can be applied to the element (think of a dynamic list view where all the items look the same from the perspective of the automation engine, but different to a real user). Or imagine a 2D or 3D game with no traditional UI controls at all---just pixels painted by a rendering engine!\n\n

\n \"A\n

\n\nHistorically, Appium hasn't tried to support these use cases, and has stuck to offering support via the XCUITests and UiAutomator2s of the world. If they couldn't find an element, Appium couldn't find an element.\n\nThere is, however, a pretty big hammer out there that could potentially be used to solve this problem. Rather like the sword of Damocles (the hammer of Damocles?) it's been hanging over the heads of the Appium developers for some time. Should we use it? Or shouldn't we? This big hammer is _visual element detection_. It's a big hammer because it uses visual features to detect elements for the purpose of interacting with them. This is exactly the same strategy that human users typically use, and therefore works no matter the kind of application and no matter whether a developer remembered to put a test ID on a particular element.\n\nThe Appium team finally decided to bite the bullet and support a small set of visual detection features, which are available as of Appium 1.9.0. In this article, we'll take a look at the most common use case for these features, namely _image element finding_. (In Part 2 of this series, we'll look at advanced methods for modulating this feature, but for now we'll stick to the basics).\n\nWhat is an image element? An image element looks to your client code exactly like any other element, except that you've found it via a new `-image` [locator strategy](https://appiumpro.com/editions/20). Instead of a typical selector (like \"foo\"), the strings used with this new locator strategy are Base64-encoded image files. The image file used for a particular find action represents a _template_ image that will be used by Appium to match against regions of the screen in order to find the most likely occurrence of the element you're looking for.\n\nThis does mean that you have to have an image on hand that matches what you want to find in the app, of course. How does Appium use this image to find your element? Under the hood, we rely on the [OpenCV](https://opencv.org/) library to find areas of the screenshot that most closely match your reference/template image. OpenCV is pretty sophisticated, and the match will succeed even despite a variety of differences between the screen and your reference image---for example differences in rotation or size.\n\nReady for an example? Let's dive in. This is a brand-new feature and both client and server behavior are a little rough around the edges. It works pretty magically, though! For the app side of things, I've created a new feature in The App which is a list of photos, displayed in some random order. When you tap on a photo, an alert pops up with a description of that photo, as in the image below:\n\n

\n \"The\n

\n\nWithout image matching, this would be an impossible scenario to automate. None of the images have any identifying information in the UI tree, and their order changes every time we load the view, so we can't hardcode an element index if we want to tap a particular image. Find-by-image to the rescue! Actually using this strategy is the same as finding an element using any other strategy:\n\n```java\nWebElement el = driver.findElementByImage(base64EncodedImageFile);\nel.click();\n```\n\nOnce we have an image element, we can click it just like any other element as well. (There are a few other commands that work on image elements, but obviously most common actions are unavailable---`sendKeys`, for example. This is because we don't actually have a reference to an actual UI element, just the region of the screen where we believe the element to be located).\n\nOf course, for this to work we have to have a Base64-encoded version of our image file. In Java 8 this is pretty straightforward:\n\n```java\n// assume we have a File called refImgFile\nBase64.getEncoder().encodeToString(Files.readAllBytes(refImgFile.toPath()));\n```\n\nThat's really all there is to it! One great thing is that finding elements by image supports both implicit and explicit wait strategies, so your tests can robustly wait until your reference image matches something on the screen:\n\n```java\nBy image = MobileBy.image(base64EncodedImageFile);\nnew WebDriverWait(driver, 10)\n .until(ExpectedConditions.presenceOfElementLocated(image)).click();\n```\n\nPutting it all together, we can now write a successful test for the scenario above: navigating to the Photo view of The App, tapping the exact photo we want, and verifying that we tapped the image by evaluating text which shows up subsequently. To find the photo, we use the reference template below:\n\n

\n \"The\n

\n\nHere's the relevant code (omitting the boilerplate for now):\n\n```java\nprivate String getReferenceImageB64() throws URISyntaxException, IOException {\n URL refImgUrl = getClass().getClassLoader().getResource(\"Edition031_Reference_Image.png\");\n File refImgFile = Paths.get(refImgUrl.toURI()).toFile();\n return Base64.getEncoder().encodeToString(Files.readAllBytes(refImgFile.toPath()));\n}\n\npublic void actualTest(AppiumDriver driver) throws URISyntaxException, IOException {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n // get to the photo view\n wait.until(ExpectedConditions.presenceOfElementLocated(photos)).click();\n\n // wait for and click the correct image using a reference image template\n By sunriseImage = MobileBy.image(getReferenceImageB64());\n wait.until(ExpectedConditions.presenceOfElementLocated(sunriseImage)).click();\n\n // verify that the resulting alert proves we clicked the right image\n wait.until(ExpectedConditions.alertIsPresent());\n String alertText = driver.switchTo().alert().getText();\n Assert.assertThat(alertText, Matchers.containsString(\"sunrise\"));\n } finally {\n driver.quit();\n }\n}\n```\n\nIn the example above you can see a helper function I wrote to get the Base64-encoded version of our reference image, which I have stored as a resource file in the project. Your technique for getting the reference image into your project might differ, of course.\n\nIn [Part 2](https://appiumpro.com/editions/33) of this series, we take a look at some of the special parameters which are available to help with finding image elements. Check it out! Also, have a look at the [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition032_Find_By_Image.java). Note that to run this successfully, you'll need to make sure you're pulling the latest Appium Java client from source (see `build.gradle` in the project, and at least Appium 1.9.0).\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Finding Elements By Image, Part 1","shortDesc":"When you just can't find an element, whether because it's built using non-standard APIs or has no uniquely identifying markers, it's possible to fall back to image matching via a template image. This technique, when used appropriately, can help overcome roadblocks to automation.","liveAt":"2018-08-29 10:00","canonicalRef":"https://www.headspin.io/blog/finding-elements-by-image-part-1"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/32-finding-elements-by-image-part-1","slug":"32-finding-elements-by-image-part-1","path":"/editions/32-finding-elements-by-image-part-1","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0031.md","num":31,"rawMd":"\n\niOS alert handling has often been a testy subject (pun! pun!). Luckily, with the current XCUITest driver used by Appium, we can connect directly to Apple-approved method for accepting or dismissing an alert, or even getting its text. Appium provides access to this behavior via the standard suite of WebDriver alert commands, for example:\n\n```java\ndriver.switchTo().alert().alert()\ndriver.switchTo().alert().dismiss()\ndriver.switchTo().alert().getText()\n```\n\nUnfortunately, the WebDriver spec is not quite as flexible as mobile apps when it comes to the design of alerts. The WebDriver spec was based around web browser alerts, which can contain at most two action buttons. Mobile apps can create native alerts which contain 3 or even more buttons! So what do you do when you need to automate the action associated with a button which is not \"OK\" or \"Cancel\"?\n\nFor example, I've added a feature to [The App](https://github.com/cloudgrey-io/the-app)'s list view which enables you to tap a third alert button when you select a cloud type:\n\n

\n \"An\n

\n\nWhen you tap this third button, another alert appears with some extra information about the cloud type:\n\n

\n \"An\n

\n\nBut back to the first alert. We want to click the \"Learn more about Cirrostratus\" alert option. In this case, we could probably use the fact that we know the name of the button to find it in the UI hierarchy. But what if we didn't know the text of the button, only that we wanted to tap the third button, whatever it contains? Luckily, Appium has a special command for handling alerts on iOS that you can use for this purpose: `mobile: alert`.\n\nThis command does allow you to accept and dismiss alerts as usual, but that's not very interesting. We're going to look at two new use cases: getting a list of the available button labels, and tapping an alert action via its label. These features together will help solve the problem under considerations. Let's take a look at an example:\n\n```java\nHashMap args = new HashMap<>();\nargs.put(\"action\", \"getButtons\");\nList buttons = (List)driver.executeScript(\"mobile: alert\", args);\n```\n\nFollowing the pattern of the other `mobile:` methods, we use `executeScript` with a special command and parameters to get the list of button labels. Once we have this list, we can do whatever we want with it. Today, we'll just try to extract the first label which isn't one of the defaults (taking care to throw an exception if there is no such label):\n\n```java\nString buttonLabel = null;\nfor (String button : buttons) {\n if (button.equals(\"OK\") || button.equals(\"Cancel\")) {\n continue;\n }\n buttonLabel = button;\n}\n\nif (buttonLabel == null) {\n throw new Error(\"Did not get a third alert button as we were expecting\");\n}\n```\n\nAt this point, we have the value of the button we want to press in the `buttonLabel` variable. We can again use `mobile: alert` to target this specific button, using a different `action` parameter than before (with a value of `\"accept\"`), and adding the new parameter `buttonLabel` to let Appium know which button we want it to tap for us:\n\n```java\nargs.put(\"action\", \"accept\");\nargs.put(\"buttonLabel\", buttonLabel);\ndriver.executeScript(\"mobile: alert\", args);\n```\n\nThat's all there is to it! Using `mobile: alert` we have access to the full set of alert actions that can be displayed in an iOS app. See the full example below to see it working in the context of my test app, or check out the code [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition031_iOS_Alerts.java)!\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.MobileElement;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport java.util.List;\nimport org.junit.After;\nimport org.junit.Before;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition031_iOS_Alerts {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.6.1/TheApp-v1.6.1.app.zip\";\n\n private IOSDriver driver;\n private WebDriverWait wait;\n\n private By listView = MobileBy.AccessibilityId(\"List Demo\");\n private By cloud = MobileBy.AccessibilityId(\"Cirrostratus\");\n\n @Before\n public void setUp() throws IOException {\n DesiredCapabilities caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"platformVersion\", \"11.4\");\n caps.setCapability(\"deviceName\", \"iPhone 6\");\n caps.setCapability(\"app\", APP);\n driver = new IOSDriver(new URL(\"http://localhost:4723/wd/hub\"), caps);\n wait = new WebDriverWait(driver, 10);\n }\n\n @After\n public void tearDown() {\n try {\n driver.quit();\n } catch (Exception ign) {}\n }\n\n @Test\n public void testCustomAlertButtons() {\n wait.until(ExpectedConditions.presenceOfElementLocated(listView)).click();\n\n WebElement cloudItem = wait.until(ExpectedConditions.presenceOfElementLocated(cloud));\n cloudItem.click();\n\n wait.until(ExpectedConditions.alertIsPresent());\n HashMap args = new HashMap<>();\n args.put(\"action\", \"getButtons\");\n List buttons = (List)driver.executeScript(\"mobile: alert\", args);\n\n // find the text of the button which isn't 'OK' or 'Cancel'\n String buttonLabel = null;\n for (String button : buttons) {\n if (button.equals(\"OK\") || button.equals(\"Cancel\")) {\n continue;\n }\n buttonLabel = button;\n }\n\n if (buttonLabel == null) {\n throw new Error(\"Did not get a third alert button as we were expecting\");\n }\n\n args.put(\"action\", \"accept\");\n args.put(\"buttonLabel\", buttonLabel);\n driver.executeScript(\"mobile: alert\", args);\n\n wait.until(ExpectedConditions.alertIsPresent());\n\n // here we could verify that the new button press worked, but for now just print it out\n String alertText = driver.switchTo().alert().getText();\n System.out.println(alertText);\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Automating Custom Alert Buttons on iOS","shortDesc":"Alerts in native mobile apps don't always work exactly the same way that alerts in web browsers do, which means we need a bit more than the WebDriver spec can give us. Appium has a special command to help manage the extra buttons that can be present in mobile alerts. In this edition we take a look at how it works on iOS.","liveAt":"2018-08-22 10:00","canonicalRef":"https://www.headspin.io/blog/automating-custom-alert-buttons-on-ios"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/31-automating-custom-alert-buttons-on-ios","slug":"31-automating-custom-alert-buttons-on-ios","path":"/editions/31-automating-custom-alert-buttons-on-ios","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0030.md","num":30,"rawMd":"\n\nIn a previous edition of Appium Pro, we saw how to [use the W3C Actions API](https://appiumpro.com/editions/29) to automate complex gestures, including drawing some (amazing) stick figure art. The same API can obviously perform simple gestures like swiping, pinching, and zooming. However, there's occasionally a downside to using these general methods, which is that they bypass the officially-recognized APIs for standard complex actions provided by the underlying mobile automation tool, for example iOS.\n\nIf you ever run into difficulty using the W3C Action API, Appium provides direct access to these vendor-supported action methods as well. In this article we'll take a look at the ones available for iOS. Because these are not part of the WebDriver spec, Appium provides this access by overloading the `executeScript` command, as you'll see in the examples below.\n\n### `mobile: swipe`\n\nThis command ultimately calls the `XCUIElement.swipe*` family of methods provided by XCUITest, and thus takes two parameters: a direction (whether to swipe up, down, left, or right), and the ID of an element within which the swipe is to take place (Appium defaults to the entire Application element if no element is specified).\n\n> **Note**: for this command and all other `mobile:` commands which have an element as a parameter, the value which should be supplied is the _internal ID_ of the element, which is not normally needed as part of Selenium/Appium testing. To get it in the Java client, you can call `element.getID()` (potentially needing to cast `element` to `RemoteWebElement` first).\n\nExample:\n\n```java\n// swipe up then down\nMap args = new HashMap<>();\nargs.put(\"direction\", \"up\");\ndriver.executeScript(\"mobile: swipe\", args);\nargs.put(\"direction\", \"down\");\ndriver.executeScript(\"mobile: swipe\", args);\n```\n\nUnfortunately, XCUITest does not provide any parameters to modify the speed or distance of the swipe. For that, use the more general Actions API.\n\n### `mobile: scroll`\n\nIf you want to try and make sure that each movement of your gesture moves a view by the height of the scrollable content, or if you want to scroll until a particular element is visible, try `mobile: scroll`. It works similarly to `mobile: swipe` but takes more parameters:\n\n* `element`: the id of the element to scroll within (the application element by default). Call this the \"bounding element\"\n* `direction`: the opposite of how direction is used in `mobile: swipe`. A swipe \"up\" will scroll view contents down, whereas this is what a scroll \"down\" will do.\n* `name`: the accessibility ID of an element to scroll to within the bounding element\n* `predicateString`: the NSPredicate of an element to scroll to within the bounding element\n* `toVisible`: if `true`, and if `element` is set to a custom element, then simply scroll to the first visible child of `element`\n\nExample:\n\n```java\n// scroll down then up\nMap args = new HashMap<>();\nargs.put(\"direction\", \"down\");\ndriver.executeScript(\"mobile: scroll\", args);\nargs.put(\"direction\", \"up\");\ndriver.executeScript(\"mobile: scroll\", args);\n\n// scroll to the last item in the list by accessibility id\nargs.put(\"direction\", \"down\");\nargs.put(\"name\", \"Stratus\");\ndriver.executeScript(\"mobile: scroll\", args);\n\n// scroll back to the first item in the list\nMobileElement list = (MobileElement) driver.findElement(By.className(\"XCUIElementTypeScrollView\"));\nargs.put(\"direction\", \"up\");\nargs.put(\"name\", null);\nargs.put(\"element\", list.getId());\ndriver.executeScript(\"mobile: scroll\", args);\n```\n\n### `mobile: pinch`\n\nTo pinch (described by a two-finger gesture where the fingers start far apart and come together) or to zoom (described by the inverse gesture where fingers start together and expand outward), use `mobile: pinch`, which calls `XCUIElement.pinch` under the hood. As with the other methods described so far, you can pass in an `element` parameter defining the element in which the pinch will take place (the entire application by default).\n\nThe only required parameter is `scale`:\n* Values between 0 and 1 refer to a \"pinch\"\n* Values greater than 1 refer to a \"zoom\"\n\nAn additional optional parameter `velocity` can be sent, which corresponds to \"the velocity of the pinch in scale factor per second\" according to [Apple's docs](https://developer.apple.com/documentation/xctest/xcuielement/1618669-pinch).\n\nExample:\n\n```java\n// zoom in on something\nMap args = new HashMap<>();\nargs.put(\"scale\", 5);\ndriver.executeScript(\"mobile: pinch\", args);\n```\n\n### `mobile: tap`\n\nThe best way to tap on an element is using `element.click()`. So why do we have `mobile: tap`? This method allows for extra parameters `x` and `y` signifying the coordinate at which to click. The nice thing is that this coordinate is either screen-relative (if an `element` parameter is not included, the default), or element-relative (if an element parameter is included).\n\nThis means that if you want to tap at the very top left corner of an element rather than dead center, you can!\n\nExample:\n\n```java\n// tap an element very near its top left corner\nMap args = new HashMap<>();\nargs.put(\"element\", ((MobileElement) element).getId());\nargs.put(\"x\", 2);\nargs.put(\"y\", 2);\ndriver.executeScript(\"mobile: tap\", args);\n```\n\n### `mobile: doubleTap`\n\nThere's more to tapping than single-tapping! And while you can certainly build a double-tap option using the Actions API, XCUITest provides a [`XCUIElement.doubleTap`](https://developer.apple.com/documentation/xctest/xcuielement/1618673-doubletap) method for this purpose, and it could presumably have greater reliability than synthesizing your own action.\n\nIn terms of parameters, you should send in _either_ an `element` parameter, with the ID of the element you want to tap, _or both_ an `x` and `y` value representing the screen coordinate you wish to tap.\n\nExample:\n\n```java\n// double-tap the screen at a specific point\nMap args = new HashMap<>();\nargs.put(\"x\", 100);\nargs.put(\"y\", 200);\ndriver.executeScript(\"mobile: doubleTap\", args);\n```\n\n### `mobile: twoFingerTap`\n\nNot to be confused with a double-tap, a two-finger-tap is a single tap using two fingers! This method has only one parameter, which is required: good old `element` (it only works in the context of an element, not a point on the screen).\n\nExample:\n\n```java\n// two-finger-tap an element (assume element object already exists)\nMap args = new HashMap<>();\nargs.put(\"element\", ((MobileElement) element).getId());\ndriver.executeScript(\"mobile: twoFingerTap\", args);\n```\n\n### `mobile: touchAndHold`\n\nMany iOS apps allow a user to trigger special behavior by tapping and holding the finger down on a certain UI element. You can specify all the same parameters as for `doubleTap` (`element`, `x`, and `y`) with the same semantics. In addition you must set the `duration` parameter to specify how many seconds you want the touch to be held.\n\nExample:\n\n```java\n// touch and hold an element\nMap args = new HashMap<>();\nargs.put(\"element\", ((MobileElement) element).getId());\nargs.put(\"duration\", 1.5);\ndriver.executeScript(\"mobile: touchAndHold\", args);\n```\n\n### `mobile: dragFromToForDuration`\n\nAnother commonly-implemented app gesture is \"drag-and-drop\". As with all of these gestures, it's possible to build a respectable drag-and-drop using the Actions API, but if for some reason this doesn't work, XCUITest has provided a method directly for this purpose. It's a [method on the `XCUICoordinate` class](https://developer.apple.com/documentation/xctest/xcuicoordinate/1615003-press), and in my opinion the name 'dragFromToForDuration' isn't the most accurate representation of it.\n\nReally, what's going on is that we're defining a start and an end coordinate, and also the duration of the _hold on the start coordinate_. In other words, we have no control over the drag duration itself, only on how long the first coordinate is held before the drag happens. What parameters do we use?\n\n* `element`: an element ID, which if provided will cause Appium to treat the coordinates as relative to this element. Absolute screen coordinates otherwise.\n* `duration`: the number of seconds (between 0.5 and 6.0) that the start coordinates should be held\n* `fromX`: the x-coordinate of the start position\n* `fromY`: the y-coordinate of the start position\n* `toX`: the x-coordinate of the end position\n* `toY`: the y-coordinate of the end position\n\nExample:\n\n```java\n// touch, hold, and drag based on coordinates\nMap args = new HashMap<>();\nargs.put(\"duration\", 1.5);\nargs.put(\"fromX\", 100);\nargs.put(\"fromY\", 100);\nargs.put(\"toX\", 300);\nargs.put(\"toY\", 600);\ndriver.executeScript(\"mobile: dragFromToForDuration\", args);\n```\n\nAnd with that our tour of the special iOS-specific gesture methods is complete! If you want to see a working example of some of the scroll and swipe functionality, check out [this article's code on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition030_iOS_Gestures.java), which makes use of a new scrolling list view added to The App!\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"iOS-Specific Touch Action Methods","shortDesc":"Sometimes using the standard Action APIs leads to undesired behavior due to the not-always-perfect mapping between the WebDriver spec and the available mobile automation technologies Appium uses. That's why Appium also makes platform-specific action methods available for 'direct' access to underlying action automation methods.","liveAt":"2018-08-15 10:00","canonicalRef":"https://www.headspin.io/blog/ios-specific-touch-action-methods"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/30-ios-specific-touch-action-methods","slug":"30-ios-specific-touch-action-methods","path":"/editions/30-ios-specific-touch-action-methods","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0029.md","num":29,"rawMd":"\n\nMobile apps often involve the use of touch gestures, sometimes in a very complex fashion. Compared to web apps, touch gestures are common and crucial for mobile apps. Even navigating a list of items requires a flick or a swipe on mobile, and often the difference between these two actions can make a meaningful difference in the behavior of an app.\n\n## A Long Time Ago...\n\nThe story of Appium's support for these gestures is as complex as the gestures themselves can be! (Feel free to skip this section if you want to get straight to the details...) Because Appium has the goal of compatibility with WebDriver, we've also had to evolve in step with some of the changes in the WebDriver protocol. When Appium was first developed, there were two different ways of getting at actions using WebDriver's existing JSON Wire Protocol. These APIs were designed for automating web browsers, driven by a mouse, so needless to say they didn't map well to the more generally gestural world of mobile app use. To make things worse, the iOS and Android automation technologies Appium was built on top of did not expose useful general gesture primitives. They each exposed their own platform-specific API, with commands like `swipe` that took different parameters and behaved differently to each other, as well as to the intention of the JSON Wire Protocol.\n\nAppium thus faced two challenges: an inadequate protocol spec, and an inadequate and variable set of basic mobile APIs provided by Apple and Google. Our approach was to implement the JSON Wire Protocol as faithfully as possible, but _also_ to provide direct access to the platform-specific APIs, via the `executeScript` command. We defined a special prefix, `mobile:`, and implemented access to these non-standard APIs behind that prefix. So, users could run commands like `driver.executeScript(\"mobile: swipe\", args)` to directly activate the iOS-specific swipe method provided by Apple. It was a bit hacky, but gave users control over whether they wanted to stick to Appium's implementation of the standard JSON Wire Protocol, or gain direct access to the platform-specific methods.\n\nMeanwhile, the Appium team learned that the Selenium project was working on a better, more general specification for touch actions. This new API was proposed as part of the new W3C WebDriver Spec which was under heavy development at the time. The Appium team then implemented this new API, and gave Appium clients another way to automate touch actions which we thought would be the standard moving forward. Unfortunately, this was an erroneous assumption. Appium was too quick to implement this new Actions spec---the spec itself changed and was recently ratified in a different incarnation than what Appium had originally supported. At least the spec is better now!\n\n## A Happy Confluence\n\nThat brings us to today. Over the years the mobile automation technologies Appium uses have themselves evolved, and clever people have uncovered new APIs that allow Appium to perform totally (or almost totally) arbitrary gestures. The [W3C WebDriver Spec](https://w3c.github.io/webdriver/webdriver-spec.html) is also now an official thing, including the most recent incarnation of the Actions API. The confluence of these two factors means that, since Appium 1.8, it's been possible for Appium to support the W3C Actions API for complex and general gestures, for example in drawing a picture (which is what we are going to do in this edition of Appium Pro).\n\nWhy do we care about the W3C API specifically? Apart from Appium's desire to match the official WebDriver standard, the Appium clients are built directly on top of the Selenium WebDriver clients. As the Selenium clients change to accommodate only the W3C APIs, that means Appium will need to support them or risk getting out of phase with the updated clients.\n\n## The Actions API\n\nThe W3C [Actions API](https://w3c.github.io/webdriver/#actions) is very general, which also makes it abstract and a bit hard to understand. Basically, it has the concept of _input sources_, many of which can exist, and each of which must be of a certain type (like key or pointer), potentially a subtype (like mouse, pen, touch), and have a certain id (like \"default mouse\"). Pointer inputs can register actions like `pointerMove`, `pointerUp`, `pointerDown`, and `pause`. By defining one (or more) pointer inputs, each with a set of actions and corresponding parameters, we can define pretty much any gesture you like.\n\nConceptually, for example, a \"zoom\" gesture consists of two pointer input sources, each of which would register a series of actions:\n\n```\nPointer 1 (type touch, id \"forefinger\")\n- pointerMove to zoom origin coordinate, with no duration\n- pointerDown\n- pointerMove to a coordinate diagonally up and right, with duration X\n- pointerUp\n\nPointer 2 (type touch, id \"thumb\")\n- pointerMove to zoom origin coordinate, with no duration\n- pointerDown\n- pointerMove to a coordinate diagonally down and left, with duration X\n- pointerUp\n```\n\nThese input sources, along with their actions, get bundled up into one JSON object and sent to the server when you call `driver.perform()`. The server then unpacks the input sources and actions and interprets them appropriately, each input source's actions being played at the same time (each action taking up one \"tick\" of virtual time, to keep actions synchronized across input sources).\n\n## Example: Let's Draw a Surprised Face\n\nLet's take a look at some actual Java code. Because the W3C Actions API is so new, there aren't a whole lot of helper methods in the Java client we can use to make our life easier. The helper methods which do exist are pretty boring, basically implementing moving to and tapping on elements, with code like:\n\n```java\nActions actions = new Actions(driver);\nactions.click(element);\nactions.perform();\n```\n\nBut this is the kind of thing we can pretty much do already, without the Actions API. What about something cool, like drawing arbitrary shapes? Let's teach Appium to draw some circles so we can play around with a \"surprised face\" picture (just to keep things simple---as an exercise to the reader it would be interesting to augment the drawing methods to be able to also draw half-circles, so that our face could be more smiley and less surprised).\n\nIf we're going to draw some circles, the first thing we'll need is some math, so we can get the coordinates for points along a circle:\n\n```java\nprivate Point getPointOnCircle (int step, int totalSteps, Point origin, double radius) {\n double theta = 2 * Math.PI * ((double)step / totalSteps);\n int x = (int)Math.floor(Math.cos(theta) * radius);\n int y = (int)Math.floor(Math.sin(theta) * radius);\n return new Point(origin.x + x, origin.y + y);\n}\n```\n\nThe idea here is that we're going to define a circle by an origin coordinate, a radius, and a number of \"steps\"---how fine-grained our circle should be. If we pass in a value of `4` for `totalSteps`, for example, our circle will actually be a square! The greater the number of steps, the more perfect a circle it will appear. Then we use the magic of Trigonometry to determine, for a given iteration (\"step\"), which point our \"finger\" should be on.\n\nNow we need to use this method to actually do some drawing with Appium:\n\n```java\nprivate void drawCircle (AppiumDriver driver, Point origin, double radius, int steps) {\n Point firstPoint = getPointOnCircle(0, steps, origin, radius);\n\n PointerInput finger = new PointerInput(Kind.TOUCH, \"finger\");\n Sequence circle = new Sequence(finger, 0);\n circle.addAction(finger.createPointerMove(NO_TIME, VIEW, firstPoint.x, firstPoint.y));\n circle.addAction(finger.createPointerDown(MouseButton.LEFT.asArg()));\n\n for (int i = 1; i < steps + 1; i++) {\n Point point = getPointOnCircle(i, steps, origin, radius);\n circle.addAction(finger.createPointerMove(STEP_DURATION, VIEW, point.x, point.y));\n }\n\n circle.addAction(finger.createPointerUp(MouseButton.LEFT.asArg()));\n driver.perform(Arrays.asList(circle));\n}\n```\n\nIn this `drawCircle` method we see the use of the low-level Actions API in the Java client. Using the `PointerInput` class we create a virtual \"finger\" to do the drawing, and a `Sequence` of actions corresponding to that input, which we will populate as we go on. From here on out we're just calling methods on our input to create specific actions, for example moving, touching the pointer to the screen, and lifting the pointer up. (In doing this we utilize some timing constants defined elsewhere). Finally, we hand the sequence off to the driver to perform! This method is a perfectly general way of drawing a circle with Appium using the W3C Actions API. But it is not yet enough to draw a surprised face. For that, we need to specify _which_ circles we want to draw, at which coordinates:\n\n```java\npublic void drawFace() {\n Point head = new Point(220, 450);\n Point leftEye = head.moveBy(-50, -50);\n Point rightEye = head.moveBy(50, -50);\n Point mouth = head.moveBy(0, 50);\n\n drawCircle(driver, head, 150, 30);\n drawCircle(driver, leftEye, 20, 20);\n drawCircle(driver, rightEye, 20, 20);\n drawCircle(driver, mouth, 40, 20);\n}\n```\n\nHere we simply define the center points of our various face components (head, eyes, and mouth), and then draw a circle of an appropriate size and with an appropriate arrangement of parts so that it kind of looks like a face. But of course all of this is only going to work if we can find an app that will recognize our gestures as an attempt to draw something! Luckily, the \"ApiDemos\" app that is freely available from Google has such a view inside of it. So we can start an Appium session on this app and navigate directly to the `.graphics.FingerPaint` activity. Once we do this, we get the masterpiece below:\n\n

\n \"Animated\n

\n\nGo ahead and check out the [full test class](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition029_W3C_Actions.java) on GitHub to see how all the boilerplate looks or to run this example yourself. Obviously, this is a toy example, but it shows the power of the Actions API to do pretty much any kind of gesture. It's certainly suitable for the typical kinds of gestures employed in most mobile apps. But there's clearly much more creativity to be unlocked here. What will you do with the Actions API? Let me know and maybe I'll showcase your creation in a future edition!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Automating Complex Gestures with the W3C Actions API","shortDesc":"The ability to automate all the actions a user could take is essential, and that extends to touch gestures like pinch, zoom, and tapping with custom durations. Appium can do all this and more, with its support of the W3C Actions API that allows the encoding of arbitrary touch input behavior.","liveAt":"2018-08-08 10:00","canonicalRef":"https://www.headspin.io/blog/automating-complex-gestures-with-the-w3c-actions-api"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/29-automating-complex-gestures-with-the-w3c-actions-api","slug":"29-automating-complex-gestures-with-the-w3c-actions-api","path":"/editions/29-automating-complex-gestures-with-the-w3c-actions-api","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0028.md","num":28,"rawMd":"\n\nUI-driven functional tests take time, and even when we've used all the tricks up our Appium sleeves, the tests will never be as fast as, say, unit tests. The ultimate solution to this is to run more than one test at a time. Think about it this way: if you have a build in which you are constantly adding tests (as you are likely to do while ensuring quality for an app), then each additional test will add directly to your build time, if you're running them one-by-one:\n\n

\n \"Running\n

\n\nLet's say, on the other hand, that for each additional test you add to your build, you unlock an additional test execution thread. In this case, you're running every single test simultaneously, and the build is only as long as the longest test!\n\n

\n \"Running\n

\n\nIt's probably not realistic to assume that you could afford thousands of test execution threads, but it illustrates the point that often the most effective optimization you can make to reduce overall build time is to parallelize test execution. Also, with the availability of cloud-based test execution, adding additional test threads becomes a problem of cost rather than technical feasibility.\n\nIn another Appium Pro article, we'll discuss all the ins and outs of parallelization from a testsuite perspective. For now, let's just learn the mechanics of running multiple Appium sessions at a time, and leave the big-picture stuff for later.\n\nThere are basically four designs for running multiple Appium sessions at a time:\n\n1. Running multiple Appium servers, and sending one session to each server\n2. Running one Appium server, and sending multiple sessions to it\n3. Running one or more Appium servers behind the Selenium Grid Hub, and sending all sessions to the Grid Hub\n4. Leveraging a cloud provider (which itself is running many Appium servers, most likely behind some single gateway)\n\n(In this article we'll explore Options 1 and 2 above, leaving Selenium Grid and cloud-based Appium testing as topics for other articles.)\n\n### Parallel Testing - Client Side\n\nRegardless of which option you choose for the Appium server side of the equation, there is also the client side to worry about. Just because you have multiple Appium servers or a cloud-based solution doesn't mean you have parallel testing---you also need a way for your test runner or test framework to kick off the tests in parallel! This ability to run tests in multiple threads (or using an event loop) is not part of Appium itself, and must be configured in your test runner itself. Each language and test runner has its own way of accomplishing this task.\n\nIn the world of Java, the most straightforward way to do this is probably with Maven and the [Surefire plugin](https://maven.apache.org/surefire/maven-surefire-plugin/), which lets you update your `pom.xml` with the following kind of configuration:\n\n```xml\n\n org.apache.maven.plugins\n maven-surefire-plugin\n \n methods\n 4\n \n\n```\n\nHere you can see that we can either parallelize based on methods or some other grouping (classes, for example). We can also specify how many test threads we want---this should of course match the number of Appium sessions we can handle.\n\n### Parallel Testing - Server Side\n\nFor most of Appium's history, Option 1 (running multiple servers) was the only way to achieve multiple simultaneous sessions. The Appium server was not set up for handling multiple simultaneous sessions so the recommended approach was to start different Appium servers listening on different ports, for example:\n\n```\nappium -p 10000 # server 1\nappium -p 10001 # server 2\n```\n\nThen, each test thread would need to adjust the URL of the Appium server to include the appropriate port, for example:\n\n```java\n@Test\npublic void testOnServer1() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:10000/wd/hub\"), capabilities);\n actualTest(driver);\n}\n\npublic void testOnServer2() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:10001/wd/hub\"), capabilities);\n actualTest(driver);\n}\n```\n\nObviously this isn't ideal from a testsuite design perspective, since we have to have some way of specifying ports differently based on different threads, meaning creating different test classes or similar. Nowadays, Option 1 is thankfully not the only option: we can also simply start multiple sessions on a single running Appium server! This means we can forget starting multiple servers on different ports, and rely on the one server running on the default port, or any port we want.\n\nUnfortunately, this isn't everything we have to worry about. Appium orchestrates a host of services under the hood to facilitate communication between different subsystems and drivers. In some cases, an Appium driver might need to use a port or a socket, and it would not be good for another session using the same driver to compete for communication on the same port or socket. There are thus a handful of capabilities that need to be included to make sure no system resource conflicts exist between your different test sessions.\n\n#### Android Parallel Testing Capabilities\n\n* `udid`: if you don't include this capability, the driver will attempt to use the first device in the list returned by ADB. This could result in multiple sessions targeting the same device, which is not a desirable situation. Thus it's essential to use the `udid` capability, even if you're using emulators for testing (in which case the emulator looks like `emulator-55xx`).\n* `systemPort`: to communicate to the UiAutomator2 process, Appium utilizes an HTTP connection which opens up a port on the host system as well as on the device. The port on the host system must be reserved for a single session, which means that if you're running multiple sessions on the same host, you'll need to specify different ports here (for example `8200` for one test thread and `8201` for another).\n* `chromeDriverPort`: in the same way, if you're doing webview or Chrome testing, Appium needs to run a unique instance of Chromedriver on a unique port. Use this capability to ensure no port clashes for these kinds of tests.\n\n#### iOS Parallel Testing Capabilities\n\n* `udid`: as with Android, we need to make sure we specify a particular device id to ensure we don't try to run a session on the same device. For simulators, udids can be found by running `xcrun simctl list devices`. Actually, if you're doing simulator testing, I should have said that _either_ the `udid` capability is required, _or_ a distinct combination of the `deviceName` and `platformVersion` capabilities is required. See below...\n* `deviceName` and `platformVersion`: if you specify a unique combination of device name and platform version, Appium will be able to find a unique simulator that matches your requirement, and in this case you don't need to specify the udid.\n* `wdaLocalPort`: just as with UiAutomator2, the iOS XCUITest driver uses a specific port to communicate with WebDriverAgent running on the iOS device. It's good to make sure these are unique for each test thread.\n* `webkitDebugProxyPort`: for webview and Safari testing on real devices, Appium runs a small service called `ios-webkit-debug-proxy`, which mediates connections on a specific port. Again, make sure multiple test threads are not trying to speak to this service on the same port.\n\n#### Option 1 Redux\n\nOne advantage of Option 1 (running multiple Appium servers) that I didn't mention before is that you can use it to abstract away all this port configuration from your testsuite, by leveraging the `--default-capabilities` server flag. Essentially, you can define unique port values on server start (rather than in desired capabilities), for example:\n\n```\nappium -p 10000 --default-capabilities '{\"systemPort\": 8200, \"udid\": \"emulator-5554\"}'\nappium -p 10001 --default-capabilities '{\"systemPort\": 8201, \"udid\": \"emulator-5556\"}'\n```\n\n(This gets us back to the problem of now having to specify different Appium server ports in our driver creation, but this problem could be avoided with the use of something like Selenium Grid)\n\n### Putting It All Together\n\nOnce we've got our client set up to fire off multiple tests at the same time, an Appium server running which can handle multiple sessions (or we're running multiple servers), and appropriately unique desired capabilities for each test thread, we're good to go. With the Gradle setup I'm using for Appium Pro, the Maven method I discussed above is not available to me. The most straightforward approach is to parallelize based on test classes. First, I need to update my `build.gradle` to include the following directive:\n\n```gradle\ntest {\n maxParallelForks = 2\n forkEvery = 1\n}\n```\n\nThis tells Gradle that I want to run in at most 2 processes, forking a new process for every test class I encounter. What I need to do next is organize my testsuite so that different classes contain different sets of desired capabilities. Because I am multiplying test classes based on the number and type of capability sets, that means each test class should inherit a base class which contains the actual logic (we don't want to duplicate that!), so some organization like the following:\n\n```\nBaseTests\n|--BaseIOSTests\n| |--IOSTests_Group1\n| |--ISOTests_Group2\n|--BaseAndroidTests\n |--AndroidTests_Group1\n |--AndroidTests_Group2\n```\n\nIn a real, well-architected build, you probably want to follow a completely different approach, one that doesn't involve hand-coding multiple classes just for the sake of getting parallel testing going, but for the sake of this article it's what we'll do. Here's an example of login test ready to be run in parallel on different iOS simulators:\n\n`Edition028_Parallel_Testing_Base.java`\n```java\npackage Edition028_Parallel_Testing;\n\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition028_Parallel_Testing_Base {\n\n protected String AUTH_USER = \"alice\";\n protected String AUTH_PASS = \"mypassword\";\n\n protected By loginScreen = MobileBy.AccessibilityId(\"Login Screen\");\n protected By loginBtn = MobileBy.AccessibilityId(\"loginBtn\");\n protected By username = MobileBy.AccessibilityId(\"username\");\n protected By password = MobileBy.AccessibilityId(\"password\");\n\n\n public void actualTest(AppiumDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(username)).sendKeys(AUTH_USER);\n wait.until(ExpectedConditions.presenceOfElementLocated(password)).sendKeys(AUTH_PASS);\n wait.until(ExpectedConditions.presenceOfElementLocated(loginBtn)).click();\n } finally {\n driver.quit();\n }\n }\n}\n```\n\n`Edition028_Parallel_Testing_iOS.java`\n```java\npackage Edition028_Parallel_Testing;\n\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition028_Parallel_Testing_iOS extends Edition028_Parallel_Testing_Base {\n\n protected String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.5.0/TheApp-v1.5.0.app.zip\";\n\n @Test\n public void testLogin_iOS() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone 8\");\n capabilities.setCapability(\"wdaLocalPort\", 8100);\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n}\n```\n\n`Edition028_Parallel_Testing_iOS_2.java`\n```java\npackage Edition028_Parallel_Testing;\n\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition028_Parallel_Testing_iOS_2 extends Edition028_Parallel_Testing_Base {\n\n protected String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.5.0/TheApp-v1.5.0.app.zip\";\n\n @Test\n public void testLogin_iOS() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.4\");\n capabilities.setCapability(\"deviceName\", \"iPhone X\");\n capabilities.setCapability(\"wdaLocalPort\", 8101);\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n}\n```\n\nNotice that both of classes with actual `@Test` annotations differ merely by a few capabilities. This could obviously be tightened up to reduce boilerplate, using an anonymous inner class design, or similar. But it illustrates the point. Now, we can run these tests in parallel by isolating them using gradle:\n\n```\ngradle cleanTest test --tests \"Edition028_Parallel_Testing_iO*\"\n```\n\nIf you run this, you'll see the Appium server handling multiple sessions at the same time, their log output interleaved. This is another reason to potentially choose multiple servers over a single server---the ability to direct log output to different files, or otherwise keep it sensibly organized for debug purposes!\n\nAnd that's basically it. Parallel testing is a deep topic and there's lots more to say about it, so stay tuned for other articles in this genre. Don't forget to check out [the full code sample](https://github.com/cloudgrey-io/appiumpro/tree/master/java/src/test/java/Edition028_Parallel_Testing) for this edition, which also includes an Android test class. You can update the `build.gradle` file to have a `maxParallelForks` of `3`, fire up an Android emulator, and run a cross-platform suite in parallel too!\n","metadata":{"lang":"Java","platform":"All Platforms","subPlatform":"All Devices","title":"Running Multiple Appium Tests in Parallel","shortDesc":"The best way to achieve a speedy build when it's full of Appium tests is to run those tests in parallel. In this article, we explore how to set up parallel testing locally, utilizing either multiple Appium servers or just a single server hosting multiple sessions.","liveAt":"2018-08-01 10:00","canonicalRef":"https://www.headspin.io/blog/running-multiple-appium-tests-in-parallel"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/28-running-multiple-appium-tests-in-parallel","slug":"28-running-multiple-appium-tests-in-parallel","path":"/editions/28-running-multiple-appium-tests-in-parallel","news":null,"tags":["Java","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0027.md","num":27,"rawMd":"\n\n> This article is the ninth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out the previous episodes on [Test Flakiness](https://appiumpro.com/editions/19), [Finding Elements Reliably](https://appiumpro.com/editions/20), [Waiting for App States](https://appiumpro.com/editions/21), [Dealing With Unfindable Elements](https://appiumpro.com/editions/22), [Setting Up App State](https://appiumpro.com/editions/23), [Advanced Capabilities](https://appiumpro.com/editions/24), [Disabling Animations](https://appiumpro.com/editions/25), and [Mocking External Services](https://appiumpro.com/editions/26).\n\nWe come finally to the last episode in this not-so-little series on test speed and stability. This edition is not about specific speed and reliability techniques; instead, it's about what to do when none of the previously-discussed suggestions have been helpful. There are no doubt lots of other useful tips hiding in the wings, and they'll make their appearance here in due time. But even if there's not an obvious solution to a particular problem, it's very useful to be able to pinpoint the problem so that you can look for a reasonable workaround or notify the appropriate authorities (Appium, Apple, Google, etc...).\n\nIn order to pinpoint any kind of problem with an Appium test (whether it's a straight-up error, or undesired slowness or instability), it's necessary to understand something about the complexity of the technology stack that comes into play during test execution. Take a look at this diagram of (most of) the pieces of Appium's XCUITest stack:\n\n

\n \"Appium\n

\n\nYou've got your test code, the Appium client code, the Appium server code, WebDriverAgent, XCUITest itself, and who knows how much proprietary Apple code shoved in between XCUITest and the various bits of iOS. A problem on any one of these layers will eventually manifest itself on the level of test behavior. It's not impossible to figure out what's going on, and there are typically lots of helpful clues lying about, but my point is that a little work is required to get to a useful answer about your problem. So, here is what I recommend doing to diagnose issues, in roughly this order:\n\n#### Test Code Exceptions\n\nThe first investigation should always be into your own test code. Oftentimes an exception arises that has nothing to do with Appium whatsoever, and is a simple programming error in your test code itself. Always read your test code exceptions in detail! Since your test code imports the Appium client in your language, this is also where you will get the first indication of any other errors. Whenever possible, Appium puts useful information in the error messages it returns to the client, and these messages should end up in your IDE or console.\n\nIn the case of exceptions generated by the Appium client, the exception will often belong to a certain class (`NoSuchElementException`, for example), which will give you a clue as to what has gone wrong. In the case of a generic exception, look for the message itself to see if the Appium server has conveyed anything about what could be wrong. Often, system configuration errors will be noticed by the Appium server and expressed in this manner, and a quick look at the message may even give you specific instructions as to how to fix the issue.\n\n#### Appium Logs\n\nThe Appium server writes a _ton_ of messages to the console (or to a file if that's how you've set it up). In the case of normal test execution, this information is mostly superfluous. It tells you all about what Appium is doing, and gives you a very detailed picture of the flow of HTTP requests/responses, commands Appium is running under the hood, and much more. Check out the [Appium Pro article on how to read the Appium logs](https://appiumpro.com/editions/10) (by Appium maintainer Isaac Murchie) for a more detailed introduction to the logs. Here are some of the things I usually look at as a first quick pass of the logs:\n\n1. The session start loglines, to double-check that all the capabilities were sent in as I expected.\n2. The very end of the log, to see the last things Appium did (especially in the case of an Appium crash).\n3. Any stacktraces Appium has written to the logs (since these will often have information directly related to an error).\n4. Any log lines rated \"warn\" or \"error\".\n\nIf nothing pops out after following those steps, I read through the logs in more detail to try and match up my test steps with loglines, and then look for the area in the log matching the problematic area in my test.\n\n#### Log Timestamps\n\nIn the case of slowness issues, it's essential to see how long different commands take. It's possible to determine this on the client side by adding time measurements before and after individual commands. Often, however, this doesn't reveal anything useful. We already know that a certain command is slow---that's why we're having trouble! But using the Appium logs, we can dig deeper and try to isolate the slowness to a particular Appium subroutine, which can be much more useful.\n\nBy default, the Appium log does not show timestamps, in order to keep the log lines short and uncluttered. But if you start the Appium server with `--log-timestamp`, each line will be prepended with a timestamp. You can use these to good effect by isolating the section of the log that matches the problematic area in your test, and start looking for large jumps in time in the log. You might find a single line or a small set of lines which are taking a long time. If it is surprising to find such a lag with those particular lines, you can always reach out to the Appium issue tracker and share your findings with the maintainers to see if there's a potential bug causing the slowness.\n\n#### Device Logs\n\nWhen it comes to problems whose root cause lies with your app, the mobile OS, or vendor-owned frameworks (like XCUITest or UiAutomator2), Appium is typically unable to provide any insight. This doesn't mean we're out of options, however. On both iOS and Android, apps can write to system-level logs, and often the system logs will also capture any exceptions thrown in your app (which might have led to a crash, say).\n\nSo, if the Appium logs don't provide any help in your investigation, the next place to go is to these logs. On Android, you can run `adb logcat` to access this log stream, and on iOS you can simply tail the system log (for iOS simulators) or use a tool like `idevicesyslog` for real devices. In fact, you can also set the `showIOSLog` capability to `true`, and Appium will scrape data from these logs and interleave it with the regular Appium logs, which often helps to put device problems into the context of the Appium session.\n\n#### Use a Different Appium Client\n\nIf you don't notice anything untoward in the Appium server logs, and suspect that the issue might lie with the Appium client you are using, you could always try to rewrite your test script (or the problematic portion of it) using a different client (either in the same language or a different language). If after doing this the problem goes away, you have pretty strong evidence that the problem was with the Appium client.\n\nHappily, the Appium team is often able to fix bugs with the Appium client very quickly (as long as the client is officially maintained by the project). Sometimes, though, a bug might exist in the underlying Selenium WebDriver client library which the Appium client is built on, and then the bug will need to be reported to the Selenium project. The Appium team can assist in that process, too.\n\n#### Use the Underlying Automation Engine Directly\n\nAppium is built on top of other automation technologies, like XCUITest as we saw in the diagram above. In many cases, issues that appear to be related to Appium are actually issues with the underlying automation technology. Before deciding that the Appium code is at fault for errors or slowness and creating a bug in the Appium issue tracker, it can be useful to write up a small testcase using the underlying engine, expressing the same test steps as with your Appium script. If you notice the same kind of problem (slowness, errors, etc...), then it's unlikely Appium will be able to do anything to directly fix the issue, and it should instead be reported to Apple or Google (and great---now you have a perfect minimal repro case for them!)\n\nIt's still useful to ask around the Appium forums in such a case, because there are often workarounds to these kinds of problems. If the workaround is useful enough, we can also build it into Appium as a first-class citizen, making life easier for all Appium users (at least until Apple or Google fixes the original problem).\n\n#### Ask for Help\n\nIf none of the strategies above have resulted in clarity on the problem or a satisfactory resolution, it's time to reach out for help. A good first step is always searching the Appium issue tracker, StackOverflow, or the Internet more generally for any sign of your problem. Maybe someone else has already discussed a workaround for it.\n\nIf that's not the case, it's a good idea to create an issue at the [Appium issue tracker](https://github.com/appium/appium/issues). At this point, if you've followed all the steps above, prepare to be showered with love and praise by the Appium maintainers, who will appreciate the extremely detailed report you will be able to include in your issue.\n\nHopefully the Appium maintainers will be able to provide a resolution, especially if it's determined that your issue reflects an Appium bug. If not, then I suppose you could always put on your crazy hacker hat and start decompiling some XCUITest source code! Yeah---that's no fun. Anyway, hopefully you'll find resolution somewhere else along the way, using the methods we've discussed here.\n\nI hope you've enjoyed this series on test speed and reliability. If in the course of your Appium test writing you come across any other good suggestions, please send them along to me, and I'll be happy to feature them in a future article. Even though the series is over, I'll certainly continue to cover tips to do with speed and reliability moving forward.\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 9: When Things Go Wrong","shortDesc":"Rounding out this series on speed and reliability, we consider failure resolution strategies. How do you debug issues? How do you locate a problem in the Appium stack? Where do you report bugs? And so on.","liveAt":"2018-07-25 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-9-when-things-go-wrong"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/27-making-your-appium-tests-fast-and-reliable-part-9-when-things-go-wrong","slug":"27-making-your-appium-tests-fast-and-reliable-part-9-when-things-go-wrong","path":"/editions/27-making-your-appium-tests-fast-and-reliable-part-9-when-things-go-wrong","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0026.md","num":26,"rawMd":"\n\n> This article is the eighth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out the previous episodes on [Test Flakiness](https://appiumpro.com/editions/19), [Finding Elements Reliably](https://appiumpro.com/editions/20), [Waiting for App States](https://appiumpro.com/editions/21), [Dealing With Unfindable Elements](https://appiumpro.com/editions/22), [Setting Up App State](https://appiumpro.com/editions/23), [Advanced Capabilities](https://appiumpro.com/editions/24), and [Disabling Animations](https://appiumpro.com/editions/25).\n\nAppium tests are usually written as end-to-end tests. In other words, they exercise the app from top (the UI) to bottom (backend services, which most apps employ). There is certainly value in this kind of testing. After all, your users are doing the same thing! Sometimes, though, what we care about with an Appium test is _not_ whether the backend service works, but whether the App Under Test itself works, on the assumption that any data retrieved from an external service is valid.\n\nThe sad truth is that the Internet is not always very reliable. Requests made by an app to an external service might fail for a number of reasons, many of which would have nothing to do with the app itself. It could be a \"stormy day\" when it comes to \"Internet weather\", and requests could time out. Or the version of the backend service used for testing might not be able to handle the load of so many requests if you're running a lot of tests at once. The list goes on.\n\nLuckily, it's possible to ensure quality for both the backend service and the mobile app in a much smarter way. The very fact that the Internet lies between these two components is the key: most backend API calls are just HTTP requests. This means that the backend service can be tested \"end-to-end\" by generating HTTP requests from something other than the app. This is called [API Testing](https://en.wikipedia.org/wiki/API_testing), and it's a _much_ faster way to test your backend service than using Appium! Similarly, all the app UI tests care about is the data that comes over the wire from the backend service: it doesn't actually matter whether it's \"real\" data or not, as long as it's an HTTP response of the appropriate form, with content appropriate to the test. So, rather than having our app make calls to a real running API service, we could have it make calls to a \"mock server\" instead!\n\nWhat are the benefits of using a mock server?\n\n1. **Speed**. Because the mock server can live on the same network as your device, API calls become blazingly fast.\n2. **Reliability**. The mock server is a very simple piece of code which just replies with canned API responses. There is very little risk of it failing. Because there is also no longer an actual Internet route in between the app and the API service it relies on, a huge source of unreliability is taken out of the picture. If the backend service itself is unreliable, the benefits increase (though that service should obviously be tested independently, so any unreliability can be addressed by the owners of the service).\n3. **Flexibility**. A mock service can even be used in the development of an app _before_ or _during_ the development of the backend service, allowing for greater flexibility and independence between teams.\n\nUsing a mock server does have some downsides, however:\n\n1. **Maintenance**. A mock server and the canned responses do have to be maintained separately of the service itself.\n2. **API Drift**. Using a mock server raises the danger that what your app is being tested against is not the same as the API the backend service is actually using. The mock server and the backend service must be kept strictly in step with one another. A good practice is to have the owner of the service also provide a mock of the service for use in testing by consumers of the service.\n3. **App and Test Complexity**. You will have to build a way for your app to select whether it makes its service calls to the real service or to the mock service, based on whether the app is being used for testing or not. You will also have to augment your tests with information about what the mock server should do, and coordinate with the app to make sure it's connecting with the mock service.\n\nWhile these downsides are real, the speed and reliability gains are definitely worth it for any sizable build. So much for an introduction! Let's see how we would go about implementing this kind of strategy. Here's what we need to get it all working:\n\n1. A mock server package of some kind (for example, the aptly named [Mock Server](http://www.mock-server.com/) for Java and JS).\n2. Changes to our app code to allow the app to connect to the mock server and not the real API server. This can be accomplished in a variety of ways, for example by having a separate build for testing which has a host and port hardcoded for the mock server, by passing startup arguments to the app to tell it where to look for its API endpoints, or by adding a new view to the app that allows setting of server details via text fields in the app UI itself.\n3. The ability to start and stop the API server before and after our testsuite, or before and after certain test classes (depending on our needs). Of course, it needs to be running on the same host and port we specified in our device.\n4. The ability for the app to talk to the mock server (this might mean ensuring the app and the mock server are on the same network).\n5. Appropriate \"expectations\" or \"scenarios\" we can set on the mock server. If we roll our own mock server, we can simply hard-code these responses in request handlers. If we're using a mock server package, it often comes with its own idiosyncratic way of specifying how the server should behave. Most often, we tell the mock server how to match against requests from the app, and how to respond based on the shape of the request.\n6. (Maybe) Depending on the mock server project we choose to incorporate into our build, it might also come with features that allow the same requests to generate different responses based on the \"scenario\". For example, we could have two different login scenarios that involve the exact same requests, one of which responds with a successful response and one of which responds with an error about e-mail formatting. In this case, we'll also need a way to uniquely identify requests from a particular device, so that requests from a device running a different test at the same time don't result in the wrong response. The best candidate for this unique id is the device id, which must be available from both the app code (so it can be sent in a header to the API server in all requests), and in the test code (so that the appropriate \"scenario\" can be selected in advance of any given test). If you want a good description of how to implement this for React Native apps specifically, check out the slides from Wim Selles's [SeleniumConf India 2018 talk](https://drive.google.com/file/d/1CRgOx2XY4JvRjWncnsEfnNZ1c0xy_APN/view).\n\nPutting all of these pieces together, the actual test flow would look like this (in pseudocode):\n\n```\nBefore:\n - Start mock server at certain port\n - Start Appium session with app (connected to server)\n\nTest:\n - Set mock server expectations\n - Use Appium to drive app, which makes calls to mock server under hood\n - Verify app behaves properly given mock server responses\n\nAfter:\n - Quit Appium session\n - Shut down mock server\n```\n\nThe Mock Server library I linked above has the ability to do all of these tasks with ease. The server can be run from the command line, as part of a maven plugin, via a Java API, etc... We can set expectations in our Java code (or other language code) using a client that comes with the library. What does setting expectations look like? Here's an example that shows how we might mock a login request:\n\n\n```java\nnew MockServerClient(\"localhost\", 1080)\n .when(\n request()\n .withMethod(\"POST\")\n .withPath(\"/login\")\n .withBody(\"{\\\"username\\\": \\\"foo\\\", \\\"password\\\": \\\"bar\\\"}\")\n )\n .respond(\n response()\n .withBody(\"{\"\\status\\\": \\\"success\\\"}\")\n );\n```\n\nRunning this command at the beginning of a test (or test suite) directs the mock server to respond with a success JSON response to any login attempt with username `foo` and password `bar`. With this kind of setup, all we need to do is direct our Appium driver to enter these values into the login prompt of our app, and the mock server will receive the request and respond with the appropriate response we've entered here. As we mentioned earlier, it's important that both the request and response match the form of the _real_ API service, otherwise we'll either run into failures or test the wrong conditions.\n\nOne difficulty in running mock API servers is encountered when utilizing a cloud service. One of the main benefits of mock servers is speed, and putting the Internet in between the app (running in the cloud) and your mock server (running locally) works against this benefit. Still, because the mock server doesn't actually have to do any work, it will be faster than a fully-fledged API server, and is worth considering even in cloud testing environments. All in all, the benefits of using mock servers are pretty compelling, and help you to reduce one of the most common types of instability in your test suite--relying on remote or 3rd-party dependencies in the form of external services.\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 8: Mocking External Services","shortDesc":"One of the most common causes of instability or non-determinacy in tests is due to dependence on external services, like a backend API server. Often, when testing your app it is not really necessary to simultaneously test these backend services, and a 'mock' server can be used instead, which greatly increases both the speed and reliability of your tests.","liveAt":"2018-07-18 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-8-mocking-external-services"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/26-making-your-appium-tests-fast-and-reliable-part-8-mocking-external-services","slug":"26-making-your-appium-tests-fast-and-reliable-part-8-mocking-external-services","path":"/editions/26-making-your-appium-tests-fast-and-reliable-part-8-mocking-external-services","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0025.md","num":25,"rawMd":"\n\n> This article is the seventh in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out the previous episodes on [Test Flakiness](https://appiumpro.com/editions/19), [Finding Elements Reliably](https://appiumpro.com/editions/20), [Waiting for App States](https://appiumpro.com/editions/21), [Dealing With Unfindable Elements](https://appiumpro.com/editions/22), [Setting Up App State](https://appiumpro.com/editions/23), and [Advanced Capabilities](https://appiumpro.com/editions/24).\n\nThe best mobile apps are designed not only to function well, but also to look good. Beyond the importance of a truly \"usable\" UI, eye candy sprinkled judiciously throughout your app has the potential to make it stand out and give that extra bit of delight to your human users. This eye candy, tasty though it might be for us humans, is completely wasted on Appium. Appium is a completely non-aesthetically-inclined robot, and doesn't care whether your app is skeuomorphic, materialed-out, or chock full of whatever weird animations the kids are into these days.\n\nThe last point is especially salient: animations take time. And unless you're directly trying to test those animations, the time spent waiting for them to complete is completely useless. Actually, it's worse than that! Animations can add instability to your testing by creating race conditions (the kind you have to work around using [explicit waits](https://appiumpro.com/editions/21) or similar). Finally, animations take up precious CPU, which can also end up reducing the reliability of your device under test. What would be great is if we could simply do away with them entirely when Appium is running its tests, since they serve no useful purpose. And we can!\n\n### Disabling Default or System Animations\n\nIf you do some spelunking on the internet, you'll find lots of articles on disabling animations for Android devices. They usually involve a series of manual steps involving turning on developer mode, and tapping through a bunch of menus. This is fine if you don't mind manually interacting with your phone or emulator, but often it's better to do things programmatically, if possible. Luckily, we can make the magic happen with a set of `adb` commands:\n\n```\nadb shell settings put global window_animation_scale 0\nadb shell settings put global transition_animation_scale 0\nadb shell settings put global animator_duration_scale 0\n```\n\nThese commands work on emulators and real devices, without requiring root. Basically, they walk through each of the system animation properties and set their values to `0`, meaning we want to turn them off completely. (To reset the settings to the default, we can run the same commands with a value of `1` in each case).\n\nIf you run these commands, then interact with your device, you'll notice that there are no more visible transitions between apps, etc... Congratulations, you sped up your Android experience!\n\nOn iOS, we're not in quite as fortunate a situation. We _can_ reduce system animations somewhat, but we can't turn them off completely, and we can't even reduce them programmatically---at least not very quickly. The way to \"disable\" animations on iOS is as follows:\n\n1. Open Settings\n2. Tap \"General\"\n3. Tap \"Accessibility\"\n4. Tap \"Reduce Motion\"\n5. Flip the switch to \"On\"\n\nAs you can tell by the name of the setting we're dealing with, we're only \"reducing\" the motion involved in certain transitions on the device. Since this all takes place in the Settings app, we can actually use Appium to take these steps by writing an Appium script to find and tap the appropriate elements. However, doing so would obviate any benefit of time saved due to minimized animations, unless you have a lot of tests and can do this only one time in the device setup routine before testing begins.\n\n### Disabling App-Specific or Custom Animations\n\nDisabling system-wide animations is a good idea, but this doesn't always catch every type of animation. Apps can define their own animations, too. To deal with your app specifically, we can make use of a tip (once again supplied by long-time Appium Pro reader [Wim](https://twitter.com/wswebcreation)) that involves custom app logic based on whether your app is being built for testing. This _does_ mean you'll need the ability to commit code to the app itself, or ask the app developers to do this.\n\nHere's how it works: most animation libraries and methods take a timing parameter, which specifies how long the animation is meant to take. What we want to end up with is a build of the app where each of those timings is specified to be zero. I won't show code examples for how to do this, because it depends on your app platform and framework (iOS, Android, React Native, etc...), but for example have a look at Apple's documentation for the [UIViewPropertyAnimator](https://developer.apple.com/documentation/uikit/uiviewpropertyanimator). The `init` method takes a `TimeInterval`.\n\nBasically, the call to a method like this should be wrapped in another function which checks to see if the app is in testing mode (via some internal flag or environment variable; it's up to you). If it is, the function will return zero for any time interval. Otherwise, it will return the developer-specified value for that animation. The net result will be that all your animations will take no time at all, effectively disabling them completely. Depending on your app framework, you might be able to come up with an even simpler and more elegant solution than this sketch.\n\nBy disabling all possible animations using these two types of technique, you'll save a lot of time and add a bit of reliability back into your build. How much time you save depends on your app and how it uses animations, but every second you shave off your tests is a second shaved off your developer cycle as a whole, which brings benefits to everyone that relies on your build.\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 7: Disabling Animations","shortDesc":"Animations in mobile apps are part and parcel of the beautiful app experience we've all come to expect. They are, however, completely useless for testing. In this edition of the 'Fast and Reliable' series, we look at how to turn them off completely, or otherwise reduce them to speed up our tests and improve test stability.","liveAt":"2018-07-11 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-7-disabling-animations"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/25-making-your-appium-tests-fast-and-reliable-part-7-disabling-animations","slug":"25-making-your-appium-tests-fast-and-reliable-part-7-disabling-animations","path":"/editions/25-making-your-appium-tests-fast-and-reliable-part-7-disabling-animations","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0024.md","num":24,"rawMd":"\n\n> This article is the sixth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out the previous episodes on [Test Flakiness](https://appiumpro.com/editions/19), [Finding Elements Reliably](https://appiumpro.com/editions/20), [Waiting for App States](https://appiumpro.com/editions/21), [Dealing With Unfindable Elements](https://appiumpro.com/editions/22), and [Setting Up App State](https://appiumpro.com/editions/23).\n\nAppium sessions are not just Appium sessions. Or at least, they don't have to be. When you start a session, you can tweak the way Appium operates by means of session initialization parameters known as \"desired capabilities\" (in Selenium parlance; in the language of the W3C WebDriver spec they are simply \"capabilities\"). We all know about capabilities (which we all love to abbreviate as \"caps\") because we have to get dirty with them just to start a test successfully, using capabilities like `platformName` or `deviceName` to let Appium know what kind of thing we're interested in automating. Appium actually supports a host of [other capabilities](https://appium.io/docs/en/writing-running-appium/caps/)---over 100 of them, actually.\n\nAll these optional caps basically tell Appium to behave differently. There are caps to tweak default timeouts, to set the language on a device before the session, to adjust the orientation, or work with a host of system-specific issues like paths to important binaries or which ports Appium should use for its internal mechanisms. Some of these capabilities are useful from the perspective of speed or reliability, and can help stabilize tests, especially if you find yourself in the specific scenario that they were designed to address. So let's take a look!\n\n## Cross-platform Capabilities\n\n#### `noReset`\n\nBy default, Appium runs a reset process after each session, to ensure that a device and app are as fresh and clean as when they first arrived. This of course is important for ensuring determinism in your test suite: you don't want the aftereffects of one test to change the operation of a subsequent test! If your tests _don't_ require this kind of clean state, and you're absolutely sure of it, you can set this capability to `true` to avoid some of the time-consuming cleaning subroutines. If you don't need the safety, take a shortcut for the sake of speed!\n\n#### `fullReset`\n\nThe opposite of the previous cap is `fullReset`, which does even _more_ cleanup than by default. Set this cap to `true` to add even more time-consuming cleaning subroutines. This might be necessary if the default cleanup is not enough to make your build reliable, and you know that the instability is due to leftover state from previous tests.\n\n#### `isHeadless`\n\nBoth iOS and Android have the concept of 'headless' virtual devices, and Appium can still work with devices that are running in this mode. Set this cap to `true` to launch a simulator or emulator in headless mode, potentially leading to a performance boost, especially if you're running in a virtualized or resource-constrained environment (such as a VM farm). Of course, in headless mode any tooling you have around taking videos or screenshots outside of Appium will not work.\n\n## Android-specific Capabilities\n\n#### `disableAndroidWatchers`\n\nThe only way to check for toast messages on Android is for the Appium UiAutomator2 driver to run a loop constantly checking the state of the device. Running a loop like this takes up valuable CPU cycles and has been observed to make scrolling less consistent, for example. If you don't need the features that require the watcher loop (like toast verification), then set this cap to `true` to turn it off entirely and save your device some cycles.\n\n#### `autoGrantPermission`\n\nSet to `true` to have Appium attempt to automatically determine your app permissions and grant them, for example to avoid system popups asking for permission later on in the test.\n\n#### `skipUnlock`\n\nAppium doesn't assume that your device is unlocked, and it should be to successfully run tests. So it installs and runs a little helper app that tries to unlock the screen before a test. Sometimes this works, and sometimes this doesn't. But that's beside the point: either way, it takes time! If you know your screen is unlocked, because you're managing screen state with something other than Appium, tell Appium not to bother with this little startup routine and save yourself a second or three, by setting this cap to `true`.\n\n#### `appWaitPackage` and `appWaitActivity`\n\nAndroid activities can be kind of funny. In many apps, the activity used to launch the app is not the same as the activity which is active when the user initially interacts with the application. Typically it's this latter activity you care about when you run an Appium test. You want to make sure that Appium doesn't consider the session started until _this_ activity is active, regardless of what happened to the launch activity.\n\nIn this scenario, you need to tell Appium to wait for the correct activity, since the one it automatically retrieves from your app manifest will be the launch activity. You can use the `appWaitPackage` and `appWaitActivity` to tell Appium to consider a session started (and hence return control to your test code) only when the package and activity specified have become active. This can greatly help the stability of session start, because your test code can assume your app is on the activity expects when the session starts.\n\n#### `ignoreUnimportantViews`\n\nAndroid has two modes for expressing its layout hierarchy: normal and \"compressed\". The compressed layout hierarchy is a subset of the hierarchy that the OS itself sees, restricted to elements which the OS thinks are more relevant for users, for example elements with accessibility information set on them. Because compressed mode generates a smaller XML file, and perhaps for other Android-internal reasons, it's often faster to get the hierarchy in compressed mode. If you're running into page source queries taking a very long time, you might try setting this cap to `true`.\n\nNote that the XML returned in the different modes is ... different. Which means that XPath queries that worked in one mode will likely not work in the other. Make sure you don't change this back and forth if you rely on XPath!\n\n## iOS-specific Capabilities\n\n#### `usePrebuiltWDA` and `derivedDataPath`\n\nTypically, Appium uses `xcodebuild` under the hood to both build WebDriverAgent and kick off the XCUITest process that powers the test session. If you have a prebuilt WebDriverAgent binary and would like to save some time on startup, set the `usePrebuiltWDA` cap to `true`. This cap could be used in conjunction with `derivedDataPath`, which is the path to the derived data folder where your WebDriverAgent binary is dumped by Xcode.\n\n#### `useJSONSource`\n\nFor large applications, it can be faster for Appium to deal with the app hierarchy internally as JSON, rather than XML, and convert it to XML at the \"edge\", so to speak---in the Appium driver itself, rather than lower in the stack. Basically, give this a try if getting the iOS app source is taking forever.\n\n#### `iosInstallPause`\n\nSometimes, large iOS applications can take a while to launch, but there's no way for Appium to automatically detect when an app is ready for use or not. If you have such an app, set this cap to the number of milliseconds you'd like Appium to wait after WebDriverAgent thinks the app is online, before Appium hands back control to your test script. It might help make session startup a bit more stable.\n\n#### `maxTypingFrequency`\n\nIf you notice errors during typing, for example the wrong keys being pressed or visual oddities you notice while watching a test, try slowing the typing down. Set this cap to an integer and play around with the value until things work. Lower is slower, higher is faster! The default is `60`.\n\n#### `realDeviceScreenshotter`\n\nAppium has its own methods for capturing screenshots from simulators and devices, but especially on real devices this can be slow and/or flaky. If you're a fan of the `libimobiledevice` suite and happen to have `idevicescreenshot` on your system, you can use this cap to let Appium know you'd prefer to retrieve the screenshot via a call to that binary instead of using its own internal methods. To make it happen, simply set this cap to the string \"idevicescreenshot\"!\n\n#### `simpleIsVisibleCheck`\n\nElement visibility checks in XCUITest are fraught with flakiness and complexity. By default, the visibility checks available don't always do a great job. Appium implemented another type of visibility check inside of WebDriverAgent that might be more reliable for your app, though it comes with the downside that the checks could take longer for some apps. As with many things in life, we sometimes have to make trade-offs between speed and reliability.\n\nWell, I hope you found something in the list above that was intriguing, or that perhaps you didn't already know about. Don't forget to check the Appium docs periodically, as many features or workarounds are made available through the ever-growing set of available capabilities!\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 6: Tuning Your Capabilities","shortDesc":"You don't have to settle for 'default Appium'. If you find yourself in a place where you're experiencing issues, or want to try a different operational mode, there are a ton of desired capabilities you can check out that modulate the way Appium works under the hood. You might find that the problem you're experiencing has a well-known workaround in the form of a desired capability you can simply plug into your driver initialization.","liveAt":"2018-07-04 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-6-tuning-your-capabilities"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/24-making-your-appium-tests-fast-and-reliable-part-6-tuning-your-capabilities","slug":"24-making-your-appium-tests-fast-and-reliable-part-6-tuning-your-capabilities","path":"/editions/24-making-your-appium-tests-fast-and-reliable-part-6-tuning-your-capabilities","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0023.md","num":23,"rawMd":"\n\n> This article is the fifth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out [Part 1: Test Flakiness](https://appiumpro.com/editions/19), [Part 2: Finding Elements Reliably](https://appiumpro.com/editions/20), [Part 3: Waiting for App States](https://appiumpro.com/editions/21), and [Part 4: Dealing With Unfindable Elements](https://appiumpro.com/editions/22).\n\n\nOne of the unfortunate realities of functional testing is the \"speed limit\": we can only go as fast as the underlying technologies let us go. Taps take time. Sending keystrokes takes time. Waiting for an application to load data from the network, or finish animations... your app takes time too. And that's kind of the point: we want functional testing to be \"high-fidelity\", meaning as close to real life as possible.\n\nThat's all well and good, except when we have multiple (or many) tests that rely on the same starting point. It's a real pain to wait for a test script to type into half a dozen boxes and tap half a dozen buttons before the meat of a test begins. Probably the most common scenario is logging in: a lot of what we test in our apps is likely behind a user login verification, which typically means finding and entering text into a few text boxes and tapping a button, not to mention waiting for the verification to take place over the network. It's not crazy to imagine that logging into an app could take as long as 20 or 25 seconds, depending on a variety of factors. Let's imagine for a moment that we have a suite of 50 tests which require logging in before they can start verifying their particular requirements. If math doesn't lie, then:\n\n

\n \"Lots\n

\n\nIn this scenario, setting up the logged-in state in our application is costing us 20 minutes of build time (which might be real time if we're running in a single thread). Ouch. More than that, we're opening ourselves up to the potential of flakiness in that particular set-up flow, by running it many more times than we need to.\n\nSadly, there might not be a way to speed up the login steps using Appium itself (once it's established that the most efficient locator strategies are already in use, etc...). That doesn't mean we can't take a shortcut, though! What if we could start our test with the app already in its logged-in state? With a little help from the app developers, we can!\n\nIn this article, we're going to take a look at 5 ways to set up app state so that you don't have to use Appium to get to the point where you need to start your test. (Of course, don't forget to leave at least _one_ test that _does_ use Appium to set up the state, so you're actually covering it. You don't want your login functionality to break because none of your tests actually walk through the flow anymore!)\n\n### Technique 1: Custom Android Activities (Android-only)\n\nIf you're developing an Android app, you might be aware of the concept of [Activities](https://developer.android.com/guide/components/activities/intro-activities). Android apps might have multiple activities, which correspond to different parts of the application which may need to be accessed directly (say from a link in an e-mail).\n\nIt's possible to design your app in such a way that starting a certain activity sets up the appropriate state in your app. Essentially, a given activity is simply a trigger for state set-up. Appium makes it possible to start arbitrary activities in your app using the `startActivity` command, and passing in the package of your app along with the desired activity as parameters, for example:\n\n```java\ndriver.startActivity(new Activity(\"com.example.app\", \".ActivityName\"));\n```\n\nYou can even use the `appActivity` desired capability to dump your session immediately into the specified activity upon start, so that from the very beginning your code will be executing in the context of that activity. Just don't forget to strip out any activities that are used for testing only in the production build of your app.\n\n### Technique 2: Custom Launch Arguments (iOS-only)\n\nA similar technique is possible with iOS, even though iOS apps do not have \"activities\". Instead, iOS apps can be launched with what are essentially command-line flags called \"launch arguments\". These flags can be read by the running application. You could easily design your application to parse the flags for a variety of data, including usernames and passwords to be used for an initial UI-less login.\n\nThe way to access this feature using Appium is via the `processArguments` capability. For example, depending on how we'd set up our app, we could imagine a schema like this:\n\n```json\n{\n \"processArguments\": \"-view login -viewparams 'my_username,my_password'\"\n}\n```\n\nOf course, the app developer would have to build the app in such a way that it knows what to do with these flags, and be sure to make the production version of the app ignore them.\n\n### Technique 3: The Test Nexus\n\nA more straightforward approach that still involves UI automation using Appium is what I call the \"Test Nexus\" technique. I saw a great example of this from [Wim Selles](https://twitter.com/wswebcreation), who put the technique to good use at his company.\n\nWhat's a Test Nexus? It's essentially a \"kitchen sink\" view in your app, only available in the test build, which contains links to a variety of other locations in your app, that come with attached state. So you could have a link that says \"Logged-in Home\", or one that reads \"Cart With 1 Item\". Tapping either of those links would trigger the app to take you to the appropriate view, with the appropriate state. This does mean the app developer would need to hook the links up to any necessary logic.\n\nThe advantage of this approach is that it is cross-platform, and easy enough for the Appium test author to simply find the right button and click on it, saving a lot of time in the process. As an example of what this Test Nexus could look like, here's a screenshot of Wim's app:\n\n

\n \"A\n

\n\n### Technique 4: Deep Linking\n\nAnother cross-platform option is so-called \"Deep Linking\". This is an OS-level feature whereby clicking on a URL with a custom scheme (i.e., a vendor-specific scheme which differs from \"http\" or \"https\") actually takes you to your application. The URL is passed to the app, and thus the app can parse it for all kinds of data which then direct the user experience.\n\nUsed for testing, it's a powerful way to set up a very flexible URL structure that can be used to direct your test to anywhere you want in your app. It does require the app developer to have registered a custom scheme with the OS, and to have created some kind of URL \"controller\" that parses the URL and actually navigates the app to the correct view with the appropriate state. For example, you could have a URL that looks like:\n\n```\nyourapp://test/login/:username/:password\n```\n\nWhich, when clicked, would open up your application in a state where the login verification has already been performed with the supplied username and password. What's great is that, using Appium, you don't need to find some way to click a link to this URL: you can use Appium's `driver.get()` command to trigger the URL to launch directly:\n\n```java\ndriver.get(\"yourapp://test/login/my_username/my_password\");\n```\n\nI won't say more about Deep Linking here, because I wrote a whole Appium Pro article on that topic already; [check it out](https://appiumpro.com/editions/7) for more info!\n\n### Technique 5: Application Backdoors\n\nI mention this last technique because it is interesting, even though there's not really an easy way to enable it within Appium today. The idea is basically that your application can expose application-internal methods to the outside world. These would be methods that are useful for testing and setting up state in your app.\n\nYour test script would then access these \"backdoors\" whenever it needed to set up state. At the moment this would have to be some kind of custom RPC channel between your test script and app, though there is an interesting proof-of-concept proposal to make something like this available within Appium itself (for Android). The idea was brought forward to the Appium community by Rajdeep Varma in his [AppiumConf 2018 talk](https://www.youtube.com/watch?v=ap4Zp6X-sWc&index=10&list=PLhr2WKNPXcZOmOf1k2aqkbMZwnZb2pSZL&t=0s), and it's certainly a promising one!\n\nIn the meantime, I recommend checking out one of the first 4 techniques. Anything you can do to save tests having to cover the same ground over and over again just to set up state is worth it, even if it involves a painful back-and-forth with your development team. Even if you don't have control over the source code yourself, I think it's a relatively easy argument to make that a little up-front effort to create these state set-up mechanisms will pay off for everyone in the long run, in terms of quicker and more reliable builds.\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 5: Setting Up App State","shortDesc":"Functional tests are kind of slow. It's sad, but it's a fact of life and something we don't need to get too concerned about if we remember that, just as in the Matrix, 'there is no spoon'! We can use a variety of techniques to set up app state directly, without having to automate the UI, so that our UI tests can test only the steps which we actually care about for a particular scenario.","newsletterTopNote":"I'm hard at work on my next HeadSpin University course, so bringing back this popular edition to your inbox this week! Also, can you help me figure some things out for HeadSpin University by taking this survey? You can win a prize (and my eternal gratitude)!","liveAt":"2018-06-27 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-5-setting-up-app-state"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/23-making-your-appium-tests-fast-and-reliable-part-5-setting-up-app-state","slug":"23-making-your-appium-tests-fast-and-reliable-part-5-setting-up-app-state","path":"/editions/23-making-your-appium-tests-fast-and-reliable-part-5-setting-up-app-state","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0022.md","num":22,"rawMd":"\n\n> This article is the fourth in a multi-part series on test speed and reliability, inspired by a webinar I gave on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out [Part 1: Test Flakiness](https://appiumpro.com/editions/19), [Part 2: Finding Elements Reliably](https://appiumpro.com/editions/20), and [Part 3: Waiting for App States](https://appiumpro.com/editions/21).\n\nOne of the sad truisms of mobile testing is that we can't always get what we want. The stack is so complex, the tools so nascent, that we are often in a position of having to try weird hacks just to get our tests to work. Appium does everything it can to canonize the most reliable weird hacks as official endpoints, so you can just go about your day writing straightforward Appium client code. But sometimes, the world doesn't cooperate.\n\nOne common occurrence in this category is when an element is clearly visible on the screen, and yet cannot be found. You've printed out the page source. You've gone hunting in Appium Desktop, and sure enough, there's no sign of the element! Why could this be? There are a number of reasons, but usually it has to do with how your app is developed. For iOS and Android, Appium is fundamentally at the mercy of the automation engines it builds on (XCUITest and UiAutomator2, respectively). These engines use the accessibility layer of the OS to determine which elements are available for automation. Oftentimes, apps which employ custom UI components have neglected to register these custom components with the accessibility service. This means that said components are essentially invisible from the perspective of XCUITest/UiAutomator2, and hence un-automatable from the perspective of Appium.\n\nIf this describes your problem, and if you have the ability to change the app source code (or can convince someone else to), the solution is actually pretty straightforward: simply turn on accessibility for your custom UI controls! Both Apple and Google mention how in their docs, but it's easy to understand why a developer might forget to do this, since it doesn't affect the functionality of the app. Anyway, you can check out the technique at these links:\n\n* For iOS: Apple's guide on [Making your iOS App Accessible](https://developer.apple.com/library/archive/documentation/UserExperience/Conceptual/iPhoneAccessibility/Making_Application_Accessible/Making_Application_Accessible.html#//apple_ref/doc/uid/TP40008785-CH102-SW4)\n* For Android: the [AccessibilityNodeProvider](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider) docs\n\nIf changing the app code isn't an option, or if there is some other reason that your element is not findable (for example, React Native doing something goofy when you set your element's position to `absolute`), you might need to pull out the big guns. (Sorry, that sounds violent: I should say, you might need to pull out the big keyboard! Because coding happens on keyboards!) Regardless of metaphor, what am I referring to? Tapping by coordinates. That's right: with Appium you can always tap the screen at any x/y position you like, and the tap will (/should) work. If the UI control you are targeting is underneath, the net result will be the same as having called a `findElement` command and then calling `click()` on the resulting object.\n\nThere's a problem, though, which is that tapping by coordinate is notoriously brittle---hence why my suggestions on how to do it right are showing up here, in our series on test speed and reliability. But first, why is tapping by coordinates brittle? For all the same reasons [XPath is probably not a good locator strategy](https://appiumpro.com/editions/20), and more:\n\n* Different versions of your app might put the element in a different place on screen\n* Different dynamic data on the screen might shove the element around even within the course of a single automation session\n* Different screen sizes might make the element fall under a different coordinate\n\nOne tantalizing solution would be to find the element _visually_, meaning by comparison with a reference image. In fact, Appium can do this! But it's pretty new and advanced, and will be the subject of a future Appium Pro edition once all the pieces are firmly in place.\n\nIn the meantime, a first stab at making tapping at a coordinate more reliable would be to ensure our coordinate itself is dynamically calculated based on conditions. For example, if we know that our app renders in completely the same proportions across different screen sizes, we can solve the last problem in the list above by storing coordinates as percentages of screen width and height in our code, rather than storing actual x and y values. Then we determine the specific coordinates for a given test run only after making a call to `driver.manage().window().getSize()`, so we can turn the percentages back into coordinates.\n\nThis is a good start, but it still doesn't help us in the case where the element might have moved around for a variety of reasons. Another idea (which I recently saw discussed as a [full blog post](http://www.wswebcreation.nl/clicking-on-an-element-with-appium-that-cant-be-found-in-the-ui-tree/) by [Wim Selles](https://twitter.com/wswebcreation)) is to find the coordinates of your unfindable element via reference to an element which _is_ findable.\n\nThe key insight of this tactic is that it's often the case that the position of the desired element is always fixed in relationship to some other element, which may be found using the normal `findElement` methods. We can use the size and position of the reference element to calculate absolute coordinates for the desired element. Of course, you'll have to observe your app's design closely to determine which features of the reference element are useful. For example, you might decide that the desired element is always 20 pixels to the right of the reference element. Or, it could be halfway between the right edge of the reference element and the right edge of the screen. Using a combination of the reference element's bounds and the screen dimensions, pretty much any position for your desired element can be targeted dynamically and reliably.\n\nLet's take a look at a concrete example using [The App](https://github.com/cloudgrey-io/the-app/). Currently, the initial screen of The App looks like this:\n\n

\n \"The\n

\n\nIn reality, I can find all the elements I care about via Accessibility ID. But let's pretend that the \"Login Screen\" list item is not accessible, and I can't see it in the hierarchy at all. What I'm going to do is gather information about the element right below it (\"Clipboard Demo\"), and use that to tap the Login Screen element dead center. I will rely on the facts I know about my app, which is that \"Login Screen\" is a list item of the same dimensions as \"Clipboard Demo\", and appears directly before it in the list. All I need to do, then, is find the midpoint of \"Clipboard Demo\", and then subtract the height of the list item, and I'll have the coordinate for the middle of \"Login Screen\". Basic math, right? So let's have a look at the code (which assumes we've got `driver` and `wait` objects all ready for us):\n\n```java\n// first, find our reference element\nWebElement ref = wait\n .until(ExpectedConditions.presenceOfElementLocated(referenceElement));\n\n// get the location and dimensions of the reference element, and find its center point\nRectangle rect = ref.getRect();\nint refElMidX = rect.getX() + rect.getWidth() / 2;\nint refElMidY = rect.getY() + rect.getHeight() / 2;\n\n// set the center point of our desired element; we know it is one row above the\n// reference element so we simply have to subtract the height of the reference element\nint desiredElMidX = refElMidX;\nint desiredElMidY = refElMidY - rect.getHeight();\n\n// perform the TouchAction that will tap the desired point\nTouchAction action = new TouchAction<>(driver);\naction.press(PointOption.point(desiredElMidX, desiredElMidY));\naction.waitAction(WaitOptions.waitOptions(Duration.ofMillis(500)));\naction.release();\naction.perform();\n\n// finally, verify we made it to the login screen (which means we did indeed tap on\n// the desired element)\nwait.until(ExpectedConditions.presenceOfElementLocated(username));\n```\n\nThis is a direct implementation of what I phrased in words above. The important commands are of course the commands for getting the bounds of the reference element (`element.getRect()`), and the commands for setting up an Appium `TouchAction` to natively tap on a coordinate. There are a few lines responsible for this latter purpose, but they're straightforward: press on a coordinate, wait for 500ms, and release. That's it!\n\nHappily, this technique is cross-platform, so the above code will run on both the iOS and Android versions of my app without modification. And it is relatively robust and reliable. It will never be as reliable as clicking on an element found by Accessibility ID, of course, because in that case we have a guarantee of calling an underlying tap method on a platform-specific element reference. In most cases, that element reference will exist even if the app is reorganized spatially. But if we are forced out into the cold dark, beyond all ability to find elements, then the technique discussed here will help you at least retain a modicum of sanity.\n\nIf you're interested in a runnable example showing the above code working on both iOS and Android, simply [check out the code for this edition](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition022_Tap_By_Coords.java)! And check back soon for the latest in this mini-series on speed and reliability.\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 4: Dealing With Unfindable Elements","shortDesc":"Sometimes, for whatever reason, we need to interact with elements that Appium can't find as elements. What can we do when there's no element to call 'click()' on? Hacky things, that's what we do. But we do the hacky things as reliably as we can.","liveAt":"2018-06-20 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-4-dealing-with-unfindable-elements"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/22-making-your-appium-tests-fast-and-reliable-part-4-dealing-with-unfindable-elements","slug":"22-making-your-appium-tests-fast-and-reliable-part-4-dealing-with-unfindable-elements","path":"/editions/22-making-your-appium-tests-fast-and-reliable-part-4-dealing-with-unfindable-elements","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0021.md","num":21,"rawMd":"\n\n> This article is the third in a multi-part series on test speed and reliability, inspired by a webinar I gave recently on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out [Part 1: Test Flakiness](https://appiumpro.com/editions/19) and [Part 2: Finding Elements Reliably](https://appiumpro.com/editions/20).\n\nUsing bad locator strategies is one cause of elements in your app not being found, and contributing to flakiness and unreliability. But even when you're using the right locator strategies, and are guaranteed to find the same element every time you look, that doesn't mean the element will be there right _when_ you look! What happens when you try to find an element which just isn't there? This is a remarkably common issue, and it's part of a more general phenomenon in functional tests: race conditions due to poor assumptions about app state.\n\n## Race Conditions\n\nA race condition is when two processes or procedures are operating simultaneously, and either one could finish before the other. This is a problem in automated test scripts because our code usually handles only one scenario. So in the alternative world where the unintuitive procedure finishes first, our test will fail.\n\nTo use our example of finding elements: our code usually implicitly assumes that the element will be present before we try to find it. In reality, the request to find an element and the app's own process of working to display the element are in a race. As human users of the app, we know how to gracefully lose the race: we simply wait! If we want to tap a button and it's not yet on the screen, we wait for it to show up (for a few seconds, anyway, then we get bored and open Twitter instead).\n\nAppium is less graceful, and does exactly what the client tells it to do, even if that means trying to find an element before it has been properly rendered on the screen. This is just one example of many possible examples in the category of test code assuming the app is in a certain state, but being proven wrong. It can be a particularly vexing problem because it might only show up infrequently, or only in CI environments. When we develop test code locally, we can often be tricked into making all kinds of assumptions about how races will resolve. Just because the app always wins a race (like we expect) when testing locally does not mean the app will behave the same in other environments.\n\n## Waiting for App States\n\nThe solution is to teach our test code to gain a bit of the grace of the human loser, and not blow up with exceptions just because the state wasn't what it expected. There are three basic ways to wait in your Appium scripts.\n\n### Static Waits\n\nWaiting \"statically\" just means applying a lot of good old `Thread.sleep` all over the place. This is the brute force solution to a race condition. Is your test script trying to find an element before it is present? Force your test to slow down by adding a static sleep! Taking the login test which is very familiar to Appium Pro readers, this is what it would look like using static waits:\n\n```java\n@Test\npublic void testLogin_StaticWait() throws InterruptedException {\n Thread.sleep(3000);\n driver.findElement(loginScreen).click();\n\n Thread.sleep(3000);\n driver.findElement(username).sendKeys(AUTH_USER);\n driver.findElement(password).sendKeys(AUTH_PASS);\n driver.findElement(loginBtn).click();\n\n Thread.sleep(3000);\n driver.findElement(getLoggedInBy(AUTH_USER));\n}\n```\n\nI chose 3 seconds as my static wait amount. Why did I choose that value? I'm not sure. It worked for me locally, and solved my race condition problems. Good enough, right? Not exactly. There are some major problems with this approach:\n\n* My test is now much longer than it needs to be (up to 9 seconds longer!), wasting my time and my build's time and therefore my team's time and my company's time. And time is money, friend! Oops.\n* I have staved off the chaos of a race condition... for now. Who's to say I won't wind up in some other scenario where 3 seconds won't be enough? What if one of the elements shows up based on a network request, and every so often the network is just a bit slower? My only recourse would be to keep increasing the static wait, thereby making the problem above even worse. And meanwhile I still can't sleep at night.\n* Unless I'm diligent at experimentation and commenting, no one will know why I picked the precise values that I did. Was it random or was there a reason?\n\nSo what else can we do?\n\n### Implicit Waits\n\nBecause the designers of Selenium were well aware of the element finding race conditions, a long time ago they added the ability in the Selenium (and we copied it with the Appium) server for the client to set an \"implicit wait timeout\". This timeout is remembered by the server and used in any instance of element finding. If an element can't be found instantly, the server will keep trying to find it up to the specified timeout. The same test implemented with implicit waits would look like:\n\n```java\n@Test\npublic void testLogin_ImplicitWait() {\n driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);\n\n driver.findElement(loginScreen).click();\n driver.findElement(username).sendKeys(AUTH_USER);\n driver.findElement(password).sendKeys(AUTH_PASS);\n driver.findElement(loginBtn).click();\n driver.findElement(getLoggedInBy(AUTH_USER));\n}\n```\n\nWow! That code is a lot nicer, for one. We've also completely solved the problem about the ever-increasing waste of time we ran into with static waits. Because the server-side element-finding retry is on a pretty tight loop, we're guaranteed to find the element within (say) a second of when it actually shows up, meaning we waste very little time while simultaneously making our test much more robust.\n\nThere's still a problem or two with this approach, however:\n\n* Using implicit wait, we tend to set one timeout and forget about it. Inevitably, this timeout becomes pretty high because it has to be high enough to account for the slowest element we could validly wait for. This means that for other elements, which we know would never take as long to show up, we still end up wasting time waiting for them. In other words, we still want our find element command to fail relatively quickly in the case where an element truly never makes an appearance. We don't want to wait for a whole minute to decide when a few seconds would have done.\n* We've been focusing on waiting for elements, which is what implicit waits are designed around. But an element's presence is just one example of an app state that we might want to wait for. What about an element's text, or visibility? Implicit wait won't help us there.\n\nThankfully, there's an even more general solution that gets us past these (admittedly more minor) issues as well.\n\n### Explicit Waits\n\nExplicit waits are just that: they make explicit what you are waiting for and how long it will take. At the cost of a little more verbosity, we get much more fine-grained control, and are able to teach our test script how to wait for just the right condition in our app before moving on. For example:\n\n```java\n@Test\npublic void testLogin_ExplicitWait() {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n wait.until(ExpectedConditions.presenceOfElementLocated(loginScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(username)).sendKeys(AUTH_USER);\n wait.until(ExpectedConditions.presenceOfElementLocated(password)).sendKeys(AUTH_PASS);\n wait.until(ExpectedConditions.presenceOfElementLocated(loginBtn)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(getLoggedInBy(AUTH_USER)));\n}\n```\n\nHere we use the `WebDriverWait` constructor to initialize a `wait` object with a certain timeout. We can reuse this object anytime we want the same timeout. We can configure _different_ wait objects with different timeouts and use them for different kinds of waiting or elements. Then, we use the `until` method on the wait object in conjunction with something called an \"expected condition\".\n\nAn expected condition is simply a special method which returns an anonymous inner class whose `apply` method will be called periodically until it returns something. The `ExpectedConditions` class has a number of useful, pre-made condition methods. What's great about explicit waits, though, is that we're not limited to what comes in the box. We can make our own!\n\n### Custom Explicit Waits\n\nIf the app state we want to wait for is particularly complex, we can always make our own expected condition. For example, let's say that the `click()` command is terribly unreliable, and often it fails, even when our element is found. So what we want is to keep retrying both the find _and_ click actions until they _both_ succeed one after the other. We could make a custom expected condition, like so:\n\n\n```java\nprivate ExpectedCondition elementFoundAndClicked(By locator) {\n return new ExpectedCondition() {\n @Override\n public Boolean apply(WebDriver driver) {\n WebElement el = driver.findElement(locator);\n el.click();\n return true;\n }\n };\n}\n```\n\nWe simply return a new `ExpectedCondition` and override the `apply` method with our particular logic. Then we can use this in our test code, for example as in this revision of the previous test:\n\n```java\n@Test\npublic void testLogin_CustomWait() {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n wait.until(elementFoundAndClicked(loginScreen));\n wait.until(ExpectedConditions.presenceOfElementLocated(username)).sendKeys(AUTH_USER);\n wait.until(ExpectedConditions.presenceOfElementLocated(password)).sendKeys(AUTH_PASS);\n wait.until(elementFoundAndClicked(loginBtn));\n wait.until(ExpectedConditions.presenceOfElementLocated(getLoggedInBy(AUTH_USER)));\n}\n```\n\nGranted, this is a bit of a useless example, but it demonstrates how easy it is to create useful and reusable waits that your whole team can use. And this completes our tour of strategies for waiting for app states with Appium. Don't forget to take a look at the [full code](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition021_Waiting.java) for the examples shown here, including all the setup and teardown boilerplate! Interested in the next episode in this series on speed and reliability? Head on over to [Part 4: Dealing with Unfindable Elements](https://appiumpro.com/editions/022).\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 3: Waiting for App States","shortDesc":"In this edition of the miniseries on speed and reliability of tests, we examine a very important aspect of writing any functional test: making sure that the app is in the state you expect before you attempt to interact with it. Here we learn how to ensure your assumptions about said state are correct.","liveAt":"2018-06-13 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-3-waiting-for-app-states"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/21-making-your-appium-tests-fast-and-reliable-part-3-waiting-for-app-states","slug":"21-making-your-appium-tests-fast-and-reliable-part-3-waiting-for-app-states","path":"/editions/21-making-your-appium-tests-fast-and-reliable-part-3-waiting-for-app-states","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0020.md","num":20,"rawMd":"\n\n> This article is the second in a multi-part series on test speed and reliability, inspired by a webinar I gave recently on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). You might also want to check out [Part 1: Test Flakiness](https://appiumpro.com/editions/19).\n\nOne of the biggest surface indicators of instability or flakiness is an element not being found. Finding elements is a natural place for problems to arise, because it is when we try to find an element that our assumption about an app's state and its actual state are brought together (whether in harmony or in conflict). We certainly want to avoid potential problems in finding elements which we can do something about, for example using selectors that are not unique, or trying to find elements by some dynamic attribute which cannot be relied on. This means that knowledge of your app and its design are essential. What is likely to change? What isn't? Which elements have accessibility IDs?\n\n## Locator Strategies\n\nBefore going further, it's worth saying what we mean by \"finding elements\", and \"accessibility ID\", for example. In Appium (as with Selenium), actions can be taken on specific objects in the app UI. These objects (corresponding to elements in a webpage, hence the name of the `findElement` API command) must be \"found\" before it is possible to interact with them. There are different ways of finding elements. Take a look at the example call below:\n\n

\n \"Components\n

\n\nIn this example, `By.className` represents a so-called \"locator strategy\" called \"class name\", and `Button` represents a \"selector\" which the strategy uses to find one or more elements. The result of this call is (if all goes well) an object of type `WebElement`, which comes with the rich set of interaction APIs you rely on for your testing.\n\n\"Class name\" is just one of a number of locator strategies available in Appium, and refers to a platform-specific UI object class name, for example `XCUIElementTypeButton` or `android.widget.Button`. Already you can see that perhaps this locator strategy isn't always ideal; what if you are testing a cross-platform app? Would you need to have a different set of code to find an iOS button or an Android button? If you rely on the \"class name\" locator strategy, the answer is yes.\n\nThere's another problem with this strategy: it's often the case that there is more than one element of any given type in the hierarchy. Thus you might very well find _a_ button with this locator strategy, but will it be _the_ button you want? So we could say that the \"class name\" locator strategy is not a good choice because it is platform-specific (leads to branched iOS and Android code), and too general (hard to uniquely identify an element with). What other options are there? Have a look at this table of the full set:\n\n

\n \"All\n

\n\nAs you can see, many of the locator strategies were carried over from Selenium, though not all are supported or even make sense in Appium (at least when automating a native app). Appium has also introduced a number of its own strategies, such as \"accessibility id\", to reflect the fact (and take advantage of the fact) that we're dealing with mobile app UIs and an entirely different automation stack.\n\n## XPath\n\nIn another Appium Pro article, I go into detail about [the XPath locator strategy, and why it should be avoided](https://appiumpro.com/editions/8). To summarize here, many people find the XPath strategy attractive, because it guarantees that any element in the UI can be found. The problem is that some elements can only be found using selectors that are \"brittle\", meaning they are liable to find no element, or a different element, if anything changes in your app's design. XPath can also be slow with Appium, because it entails sometimes multiple recursive renderings of the UI hierarchy.\n\n## Accessibility ID\n\nWhat should we use instead? When possible, I recommend using the \"accessibility ID\" locator strategy, because it is (a) cross-platform, (b) unique, and (c) fast. Both iOS and Android have the concept of an accessibility label, though on iOS it's called \"accessibility ID\" and on Android it's called \"content description\" (or \"content-desc\"). Since the accessibility label is a string set by developers, it can be a unique identifier. Of course, the point of setting accessibility labels is for users, not for testers, so care must be taken. Setting good user-centric accessibility labels is usually sufficient to provide the kinds of identifiers testers also need, and that is the best practice (i.e., set accessibility labels on everything for users, and then take advantage of this for testing). In the Appium Java client, finding elements by accessibility ID involves using the `MobileBy` strategy:\n\n```java\nWebElement el = driver.findElement(MobileBy.AccessibilityID(\"foo\"));\n```\n\nSince testers don't always have the ability to influence the app's development, sometimes accessibility labels are not available, or are not unique. What else could we use?\n\n## iOS-specific Locator Strategies\n\nIn the same Appium Pro article I referenced earlier, I went into detail on some [iOS-specific locator strategies](https://appiumpro.com/editions/8) that could be used as a substitute for XPath, because they are hierarchical query-based strategies. The most robust is the \"-ios class chain\" strategy, which allows you to use a \"lite\" version of something like XPath, mixed together with iOS [predicate format strings](https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/Predicates/Articles/pSyntax.html).\n\nThe benefit of this locator strategy is that it allows for complex queries while remaining in most cases much speedier than XPath. The drawback, of course, is that it is platform-specific, so requires branching your code (or adding further distinctions to your object models). As an example of what you can do, check out this command:\n\n```java\nString selector = \"**/XCUIElementTypeCell[`name BEGINSWITH \"C\"`]/XCUIElementTypeButton[10]\";\ndriver.findElement(MobileBy.iOSClassChain(selector));\n```\n\nWhat we're doing here is finding the 10th button which is a child of a table cell anywhere in the UI hierarchy which has a name beginning with the character \"C\". That's quite the query! Because of the more rigid form of class chain queries, the performance guarantees are better than those of XPath.\n\n## Android-specific Locator Strategies\n\nA similar trick is available for Android, in the guise of a special parser the Appium team implemented which supports most of the [UiSelector](https://developer.android.com/reference/android/support/test/uiautomator/UiSelector) API. We make this parser available via the \"-android uiautomator\" locator strategy, and the selectors should be strings which are valid bits of Java code beginning with `new UiSelector()`. Let's have a look at an example:\n\n```java\nString selector = \"new UiSelector().className(\\\"ScrollView\\\").getChildByText(new UiSelector().className(\\\"android.widget.TextView\\\"), \\\"Tabs\\\")\";\ndriver.findElement(MobileBy.AndroidUIAutomator(selector));\n```\n\nOnce again, we make use of a `MobileBy` strategy since this strategy is available only for Appium. What's going on here is that we have constructed a string which could be used as valid UiAutomator test code, but in fact will be parsed and interpreted by Appium when the command is sent. According to the semantics of the UiSelector API, we're saying that we want the first `TextView` element we find with the text \"Tabs\", which is also a child of the first `ScrollView` in the hierarchy. It's a bit clunkier than XPath, but it can be used in similar ways, and again with a better performance profile in most cases.\n\nAs with the iOS class chain strategy, the main downside here is that selectors are going to be platform-specific. (Additionally, we can't support arbitrary Java and there are limits to what we can provide from the UiSelector API).\n\n## Determining Which Selectors to Use\n\nSo far we've seen some good recommendations on which strategies to use to find elements reliably. But how do you know which selectors to use in conjunction with those strategies? I said before that knowledge of your app is required in order to do this correctly. How do you get that knowledge of your app? If you're one of the app developers, you can simply have a look at the code, or maybe you remember that you gave a certain element a certain accessibility label. If you don't have access to the code, or if you want a method that will show you exactly what Appium sees in your app, then it's best to use [Appium Desktop](https://github.com/appium/appium-desktop).\n\nAppium Desktop is a GUI tool for running Appium and inspecting apps. You can use it to launch \"inspector sessions\" with arbitrary desired capabilities. Inspector sessions show you a screenshot of your app, its UI hierarchy (as XML), and lots of metadata about any element you select. It looks like this:\n\n

\n \"The\n

\n\nOne of the great things about the Inspector is that, when you click on an element in the hierarchy, it will intelligently suggest locator strategies and selectors for you. In the image above, you can see that the top suggestion for the selected element is the \"accessibility id\" locator strategy, used in conjunction with the selector \"Login Screen\".\n\nThings can get a bit more complex, certainly, but the Appium Desktop Inspector is always a great place to start when figuring out what's going on with your app hierarchy. It's especially useful if you run into issues where you think an element should exist on a certain view: just fire up the Inspector and manually look through the XML tree to see if in fact the element exists. If it doesn't, that means Appium (read: the underlying automation frameworks) can't see it, and you'll need to ask your app developer why.\n\nAnd that concludes our discussion of finding elements reliably in Appium---or at least one aspect of it. Just because you can find an element with the correct locator strategy doesn't mean it will always be there when you look. Make sure to also check out the next part, on [waiting for app states](https://appiumpro.com/editions/21) (including the presence of elements).\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 2: Finding Elements","shortDesc":"Continuing a larger Appium Pro series on making tests more robust, in this episode we examine the various locator strategies available within Appium, when to use which, and other approaches and practices to making sure your elements are found and not lost.","liveAt":"2018-06-06 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-2-finding-elements"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/20-making-your-appium-tests-fast-and-reliable-part-2-finding-elements","slug":"20-making-your-appium-tests-fast-and-reliable-part-2-finding-elements","path":"/editions/20-making-your-appium-tests-fast-and-reliable-part-2-finding-elements","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0019.md","num":19,"rawMd":"\n\nLet's face it, Appium tests have sometimes been accused of being slow and unreliable. In some ways the accusation is true: there are fundamental speed limits to the automation technologies Appium relies on, and in the world of full-fledged functional testing there are a host of environmental problems which can contribute to test instability. In other ways, the accusation is misplaced, because there are strategies we can use to make sure our tests don't run into common pitfalls.\n\nThis article is the first in a multi-part series on test speed and reliability, inspired by a webinar I gave recently on the same subject (you can [watch the webinar here](https://www.youtube.com/watch?v=Sx-jscrsSRM&list=PLFoVxiw64PraVz_N33Z4uv-GzMmLm7EBI)). The webinar was so jam-packed with content that I barely had the opportunity to catch my breath in between topics and I still went over time! So in this series we're going to take each piece a little slower and in more detail. For this first part, we'll discuss the blood-pressure-raising notion of test flakiness.\n\n## \"Flakiness\"\n\nNo discussion of functional test reliability would be complete without addressing the concept of \"flakiness\". According to common usage, \"flakey\" is synonymous with \"unreliable\"---the test passes some times and fails other times. The blame here is often put on Appium---if a test passes once when run locally, surely any future failures are due to a problem with the automation technology? It's a tempting position to take, especially because it takes us (the test authors) and our apps out of the crosshairs of blame, and allows us to place responsibility on something external that we don't control (scapegoat much?).\n\nIn reality, the situation is much more complex. It may indeed be the case that Appium is responsible for unreliable behavior, and regrettably this does happen in reality. But without an investigation that _proves_ this to be the case for your particular test, the problem may with equal probability lie in any number of other areas, for example:\n\n* Unwarranted assumptions made by the test author about app or device speed, app state, screen size, or dynamic content\n* App instability (maybe the app itself exhibits erratic behavior, even when used by a human!)\n* Lack of compute or memory resources on the machine hosting a simulator/emulator\n* The network (sometimes HTTP requests to your backend just fail, due to load or issues outside of your team's control)\n* The device itself (as we all know, sometimes real devices just do odd things)\n\nFurthermore, even if it can be proved that none of these areas are problematic, and therefore \"Appium\" is responsible, what does that mean? Appium is not one monolithic beast, even though to the user of Appium it might look that way. It is in fact a whole stack of technologies, and the erratic behavior could exist at any layer. To illustrate this, have a look at this diagram, showing the various bits of the stack that come into play during an iOS test:\n\n

\n \"XCUITest\n

\n\nThe part of this stack that the Appium team is responsible for is really not that deep. In many cases, the problem lies deeper, potentially with the automation tools provided by the mobile vendors (XCUITest and UiAutomator2 for example), or some other automation library.\n\nWhy go into all this explanation? My main point isn't to take the blame away from Appium. I want us to understand that when we say a test is \"flakey\", what we really mean is \"this test sometimes passes and sometimes fails, and I don't know why\". Some testers are OK stopping there and allowing the build to be flakey. And it's true that some measure of instability is a fact of life for functional tests. But I want to encourage us not to stick our heads in the sand---the instability we can't do anything about is relatively small compared to the flakiness we often settle for out of an avoidance of a difficult investigation.\n\nMy rule of thumb is this: only allow flakey tests whose flakiness is _well understood_ and _cannot be addressed_. This means, of course, that you may need to get your hands dirty to figure out what exactly is going on, including coming to the Appium team and asking questions when it looks like you've pinned the problem down to something in the automation stack. And it might mean ringing some alarm bells for your app dev team or your backend team, if as a result of your investigation you discover problems in those areas. (One common problem when running many tests in parallel, for example, is that a build-only backend service might be underpowered for the number of requests it receives during testing, leading to random instability all over. The solution here is either to run fewer tests at a time, or better yet, get the backend team to beef up the resources available to the service!)\n\nLike any kind of debugging, investigations into flakey tests can be daunting, and are led as much by intuition as by method. If you keep your eyes open, however, you will probably make the critical observations that move your investigation forward. For example, you might notice that a certain kind of flakiness is not isolated to one test, but rather pops up across the whole build, seemingly randomly. When you examine the logs, you discover that this kind of flakiness always happens at a certain time of day. This is great information to take to another team, who might be able to interpret it for you. \"Oh, that's when this really expensive cron job is running on all the machines that host our Android emulators!\", for example.\n\nWe'll dig into the topic of debugging failed tests in a future part in this series. For now, my concrete recommendation for handling flakiness in a CI environment in general is as follows:\n\n1. Determine whether a test is flakey _before_ permanently adding it to your build. In a perfect world, this would look like automatically catching any functional test that's being committed, and running it many times (maybe 100?) to build a reliability profile for it. If it passes 100% of the time, great! Merge that commit to master and off you go.\n2. If the test doesn't always pass, it's unreliable or flakey. But we can't stop there, because \"flakey\" is a codeword for ignorance. Time to dig in and find out why it's unreliable. Usually with a little investigation it's possible to see what went wrong, and perhaps adjust an element locator or an explicit wait to handle the problem. At this stage, Appium logs and step-by-step screenshots are essential.\n3. Once you discover the cause of flakiness, you'll either be able to resolve the flakiness or not. If you can do something to resolve it, it is incumbent on you to do so! So get that test passing 100% of the time. If you determine that there's nothing you can do about it (and no, filing a bug report to Appium or to Apple is not \"nothing\"), you have two options: either forfeit the test if it's not going to provide more value than headache in the long run, or annotate it such that your CI system will retry the test once or twice before considering it failed. (Or only run it before a release, when you have time to manually verify whether a failure is a \"flake\").\n4. If you take the approach of keeping the test in your build and allowing the build to retry it on failure, you _must_ track statistics about how much each test is retried, and have some reliability threshold above which a new investigation is triggered. You don't want tests creeping up in flakiness over time, because that could be a sign of a real problem with your app.\n\nKeep in mind that Appium tests are functional tests, and not unit tests. Unit tests are hermetically sealed off from anything else, whereas functional tests live in the real world, and the real world is much more messy. We should not aim for complete code coverage via functional testing. Start small, by covering critical user flows and getting value out of catching bugs with a few tests. Meanwhile, make sure those few tests are as rock solid as possible. You will learn a lot about your app and your whole environment by hardening even a few tests. Then you'll be able to invest that learning into new tests from the outset, rather than having to fix the same kinds of flakiness over and over again down the road.\n\nReady for more test robustness? Head on over to [Part 2](https://appiumpro.com/editions/20), where we'll talk about quickly and reliably finding elements in your apps!\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Making Your Appium Tests Fast and Reliable, Part 1: Test Flakiness","shortDesc":"In this article we start a series that takes a look at a collection of tips and best practices that all contribute to greater speed and reliability of your Appium tests. First off, we discuss the dreaded concept of test 'flakiness', and how we should approach this concept within the world of Appium.","liveAt":"2018-05-30 10:00","canonicalRef":"https://www.headspin.io/blog/making-your-appium-tests-fast-and-reliable-part-1-test-flakiness"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/19-making-your-appium-tests-fast-and-reliable-part-1-test-flakiness","slug":"19-making-your-appium-tests-fast-and-reliable-part-1-test-flakiness","path":"/editions/19-making-your-appium-tests-fast-and-reliable-part-1-test-flakiness","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0018.md","num":18,"rawMd":"\n\n[Espresso](https://developer.android.com/training/testing/espresso/) is an Android test automation library maintained by Google. It has a number of advantages, for example built-in view synchronization that ensures element finding happens during idle periods in your app. (Curious to learn more about Espresso and how it compares to Appium and other frameworks? Check out my webinar, [The Shifting Landscape of Mobile Automation](https://www.youtube.com/watch?v=AV8p2aeqsOg) for a more in-depth discussion). Most people assume Espresso is an alternative to Appium; you'd pick _either_ Appium _or_ Espresso, not both. That's pretty much been the case, up until now.\n\nPart of Appium's vision is the incorporation of any good technology into Appium itself. Our goal with Appium is not to compete on fundamental automation technology, when what exists is good. Instead, our vision is of a single, unified API for all app automation, based on the official WebDriver spec. As I put it in the aforementioned webinar, here's what I think the Appium of the future looks like:\n\n

\n \"The\n

\n\nFrom this perspective, there's nothing odd about creating an Appium Espresso driver. And that is exactly what we've done. For some time we've been working on a very rough beta (really alpha) version of an Appium driver that runs Espresso under the hood. This means the same kind of Appium script you're used to writing, but running on Google's top-tier automation technology. This new driver is still in its infancy, and is not recommended for production. However, the beta is moving along enough that I think it's time more people had a chance to play with it. In this article, we'll see exactly how to do that!\n\nIn essence, it's pretty simple: we just change the `automationName` capability to `Espresso` (instead of, say, `UiAutomator2` if you're using the current standard Android driver for Appium). By designating this automation name, Appium will know to start an Espresso session instead of something else. The Espresso driver is so new, however, that you will really need to be running Appium from source (by cloning the [GitHub project](https://github.com/appium/appium) and running `npm install` to get the latest dependencies, including the latest Espresso driver), or running the latest Appium beta (`npm install -g appium@beta`). At this point, simply running `appium` (or `node .` if running from source) will spin up an Appium server that knows about the most recent Espresso driver beta.\n\nThe best way to show off what you can currently do with the Espresso beta is with a comparison. The code for this article is therefore the same test (of a basic login flow) run on both UiAutomator2 and Espresso drivers. Let's take a look at the code for the standard UiAutomator2 driver first:\n\n```java\n@Test\npublic void testLogin_UiAutomator2() throws MalformedURLException {\n AndroidDriver driver = getDriver(\"UiAutomator2\");\n WebDriverWait wait = new WebDriverWait(driver, 10);\n ExpectedCondition loginScreenReady =\n ExpectedConditions.presenceOfElementLocated(loginScreen);\n ExpectedCondition usernameReady =\n ExpectedConditions.presenceOfElementLocated(username);\n ExpectedCondition verificationReady =\n ExpectedConditions.presenceOfElementLocated(verificationTextUiAuto2);\n\n try {\n wait.until(loginScreenReady).click();\n wait.until(usernameReady).sendKeys(\"alice\");\n driver.findElement(password).sendKeys(\"mypassword\");\n driver.findElement(loginBtn).click();\n wait.until(verificationReady);\n } finally {\n driver.quit();\n }\n}\n```\n\nThe first thing to observe about this snippet is that we have a helper method, `getDriver`, which simply takes the automation name and gets us an instance of `AndroidDriver`. This is so we can reduce code duplication when we do the same thing for the Espresso version of the test (and to show that all the capabilities are the same, other than `automationName`). Next, we set up 3 expected conditions for user later on in the test. Finally, our test flow itself is 5 steps long, utilizing pre-defined locator fields like `password`. The steps themselves should be familiar from other articles: (1) get to the login prompt, (2) enter the username, (3) enter the password, (4) tap the log in button, and (5) verify that an element with the correct logged-in text is present.\n\nSo far, so good! Now let's take a look at the same test, but written for the Espresso driver:\n\n```java\n@Test\npublic void testLogin_Espresso() throws MalformedURLException {\n AndroidDriver driver = getDriver(\"Espresso\");\n WebDriverWait wait = new WebDriverWait(driver, 10);\n ExpectedCondition loginScreenReady =\n ExpectedConditions.presenceOfElementLocated(loginScreen);\n\n try {\n wait.until(loginScreenReady).click();\n driver.findElement(username).sendKeys(\"alice\");\n driver.findElement(password).sendKeys(\"mypassword\");\n driver.findElement(loginBtn).click();\n driver.findElement(verificationTextEspresso);\n } finally {\n driver.quit();\n }\n}\n```\n\nCan you spot the differences? There are just two:\n\n1. We only need 1 explicit wait instead of 3\n2. We have a different verification element we're looking for in the last step\n\nOtherwise, the test code is exactly the same! This is great, because it means that, for the most part, changes were not required to migrate this particular test to the Espresso driver. Now, why do we only need 1 explicit wait instead of 3 as before? We needed them in the UiAutomator2 example because any time we try to find an element after a view transition, we have no guarantees about the timing of when the new view will show up, and we have to hedge our bets with an explicit wait. One of the benefits of Espresso, however, is _synchronization_, which as I explained before means that Espresso itself will hold off on finding any elements until it believes the app is in an idle state. What this means is that, for the most part, we don't need to worry about waits in Espresso! (We do still need the first wait because synchronization is not in effect until the app itself is fully loaded and instrumented by Espresso, and Appium doesn't know exactly when that happens).\n\nThe second difference we mentioned was that we needed a different verification locator. What are these two locators and how do they differ? Here is how they are defined as fields on the test class:\n\n```java\nprivate By verificationTextEspresso = By.xpath(\n \"//com.facebook.react.views.text.ReactTextView[@text='You are logged in as alice']\");\n\nprivate By verificationTextUiAuto2 = By.xpath(\n \"//android.widget.TextView[contains(@text, 'alice')]\");\n```\n\nInterestingly, the Espresso driver has access to app-internal class names. We can tell, for example, that I used React Native to develop the test application, whereas with UiAutomator2, all we know is that we have a text view of some kind. This specificity in the Espresso driver is nice, but it comes potentially at a cost of reducing the cross-platform nature of the element class names. The Appium team will be looking into ways to sort this out as we move forward with work on the Espresso driver beta. Meanwhile, we also note that if we wanted, we could have written a more general XPath query that works across both drivers (something like `//*[contains(@text, 'alice')]`).\n\nOther than the view change synchronization, are there any other benefits to using the Espresso driver? In my experiments so far, it appears to be about 25% faster than the UiAutomator2 driver, though it would take a fair amount of work to ensure a \"clean-room\" environment for the experiment and corroborate that figure.\n\nSo, if you like living on the cutting edge of Appium and mobile automation, I encourage you to check out the Espresso driver. No doubt you will find that it doesn't work quite as you expect in one (or many) ways. That's part of why I'm directing you to try it; I think we're in a state now where we could really use some solid feedback and bug reports! So fire it up and let us know on the Appium issue tracker if you encounter any issues. You can also [follow along with the Espresso driver development](https://github.com/appium/appium-espresso-driver) on GitHub.\n\nThe full code for the comparison tests we looked at in this article is below, and as always can be found [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition018_Espresso_Beta.java) as well.\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.AndroidDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedCondition;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition018_Espresso_Beta {\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.5.0/TheApp-v1.5.0.apk\";\n private By loginScreen = MobileBy.AccessibilityId(\"Login Screen\");\n private By username = MobileBy.AccessibilityId(\"username\");\n private By password = MobileBy.AccessibilityId(\"password\");\n private By loginBtn = MobileBy.AccessibilityId(\"loginBtn\");\n private By verificationTextEspresso = By.xpath(\n \"//com.facebook.react.views.text.ReactTextView[@text='You are logged in as alice']\");\n private By verificationTextUiAuto2 = By.xpath(\n \"//android.widget.TextView[contains(@text, 'alice')]\");\n\n private AndroidDriver getDriver(String automationName) throws MalformedURLException {\n\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", automationName);\n capabilities.setCapability(\"app\", APP);\n\n return new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n }\n\n @Test\n public void testLogin_Espresso() throws MalformedURLException {\n AndroidDriver driver = getDriver(\"Espresso\");\n WebDriverWait wait = new WebDriverWait(driver, 10);\n ExpectedCondition loginScreenReady =\n ExpectedConditions.presenceOfElementLocated(loginScreen);\n\n try {\n wait.until(loginScreenReady).click();\n driver.findElement(username).sendKeys(\"alice\");\n driver.findElement(password).sendKeys(\"mypassword\");\n driver.findElement(loginBtn).click();\n driver.findElement(verificationTextEspresso);\n } finally {\n driver.quit();\n }\n }\n\n @Test\n public void testLogin_UiAutomator2() throws MalformedURLException {\n AndroidDriver driver = getDriver(\"UiAutomator2\");\n WebDriverWait wait = new WebDriverWait(driver, 10);\n ExpectedCondition loginScreenReady =\n ExpectedConditions.presenceOfElementLocated(loginScreen);\n ExpectedCondition usernameReady =\n ExpectedConditions.presenceOfElementLocated(username);\n ExpectedCondition verificationReady =\n ExpectedConditions.presenceOfElementLocated(verificationTextUiAuto2);\n\n try {\n wait.until(loginScreenReady).click();\n wait.until(usernameReady).sendKeys(\"alice\");\n driver.findElement(password).sendKeys(\"mypassword\");\n driver.findElement(loginBtn).click();\n wait.until(verificationReady);\n } finally {\n driver.quit();\n }\n\n }\n}\n```\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Using Espresso With Appium","shortDesc":"Espresso and Appium?! That's right, you can access the power and reliability of Google's flagship Android automation technology from within Appium, retaining the ability to write WebDriver-compatible scripts in any programming language.","liveAt":"2018-05-23 10:00","canonicalRef":"https://www.headspin.io/blog/using-espresso-with-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/18-using-espresso-with-appium","slug":"18-using-espresso-with-appium","path":"/editions/18-using-espresso-with-appium","news":null,"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0017.md","num":17,"rawMd":"\n\nIn these articles we tend to focus a lot on native mobile apps, and for good reason: automating native apps is a big reason people turn to Appium. One of Appium's greatest benefits, however, is that you can automate other varieties of app just as easily, using the same sort of client code---that's the beauty of building on top of the standard WebDriver protocol!\n\nHybrid apps are one of these \"other varieties\" of app, and they are unique in consisting of both native and web components. Hybrid apps are developed for a number of reasons, but one often-cited reason is to allow app developers the freedom to use the tools, languages, frameworks, and development process familiar from the web, while still producing an app that looks and feels right on a phone. The secret is in the \"webview\", a native component whose sole purpose is to display web content (either stored locally as HTML, JS, and CSS, or retrieved from some URL).\n\nTo demonstrate some of the interesting possibilities of hybrid apps, I've released [v1.5.0 of TheApp](https://github.com/cloudgrey-io/the-app/releases/tag/v1.5.0), which looks like this:\n\n

\n \"Screenshot\n

\n\nWe have a few native controls (a text field and two buttons), and then some text directing us to go to a webpage. In fact, this text is itself some simple HTML, hosted in a webview with an invisible frame. If I enter a URL in the text box and click \"Go\", one of two things will happen:\n\n* If I've elected to visit `https://appiumpro.com`, the request will go through and the webview will load the Appium Pro site\n* If I try any other URL, I will get a native alert telling me I'm not allowed to navigate\n\nEssentially, what I've built is a little web browser that only lets the user go to one site! (Why did I pick Appium Pro's site? Hmm....). OK, this is admittedly very useless (not to mention ugly on top). But it highlights the interactions possible between native components and the webview. Let's imagine a simple test scenario to prove this feature works:\n\n1. Navigate to the Hybrid Demo view (the one pictured above)\n2. Assert that there is a page loaded in the webview which is _not_ Appium Pro to begin with\n3. Attempt to visit some URL which is also _not_ Appium Pro (say Google)\n4. Assert that an alert pops up\n5. Attempt to navigate to Appium Pro's URL\n6. Assert that the title of the page inside the webview matches Appium Pro\n\nThe only steps we really need to cover in this guide are #3 and #6, the ones that are making a verification inside a webview. How do we do this? Every Appium session comes with the ability to switch between multiple \"contexts\". In a test of a native app, there is usually just one context available, and thus there's no point in worrying about contexts at all.\n\nIn hybrid apps, there are multiple contexts: one that represents the native portion of the app, and one or more contexts that represent each webview which is present (typically just one). With our Appium client, we can either get a list of the available contexts, or we can tell Appium to switch to a given context, using the Context API:\n\n```java\ndriver.getContextHandles(); // get a list of the available contexts\ndriver.context(\"NATIVE_APP\"); // switch to the native context\n```\n\nIn the example above, I show how to switch to the native context. But that's where we start out already, so it's not very useful. How do we switch to a webview context? Well, we don't know the name of a webview context before a session starts, so we have to use the `getContextHandles` command to figure out the correct string that refers to a webview context. I recommend using a little helper method like the following to do this without relying on any knowledge about the number of webview contexts present:\n\n```java\n@Nullable\nprivate String getWebContext(AppiumDriver driver) {\n ArrayList contexts = new ArrayList(driver.getContextHandles());\n for (String context : contexts) {\n if (!context.equals(\"NATIVE_APP\")) {\n return context;\n }\n }\n return null;\n}\n```\n\nBasically, we just loop through all the available contexts and return the first one we find which is not the native context. This simple logic is good for most cases. We can then use the output of this method to get into a webview context using the `context` method:\n\n```java\nString webContext = getWebContext(driver);\ndriver.context(webContext);\n```\n\nOnce this command completes successfully, we're in a webview! Great. But what does that mean exactly? What it means is that, from this point on, any command we send from our Appium client is going to be taken to refer to the webpage inside the webview, rather than the native app. This means that what we are dealing with at this point is essentially a _Selenium_ session. We can issue commands like `getTitle()` and get the title of the page inside the webview---something that would not work if we were in the native context! Element finding, clicks, etc..., all operate on the page inside the webview just as though you had fired up Selenium and were automating a desktop browser. (Certain device-level commands that don't make sense inside a webview continue to be interpreted in the native context, for example changing orientation).\n\nOf course, at some point you might want to target native elements or behaviors again, in which case you simply have to re-run the `context` command with the value `NATIVE_APP` to tell Appium you want to go back to native mode. Armed with this knowledge, let's see how we would implement the test scenario we sketched out above:\n\n```java\nWebDriverWait wait = new WebDriverWait(driver, 10);\nfinal String title = \"Appium Pro: The Awesome Appium Tips Newsletter\";\n\nwait\n .until(ExpectedConditions.presenceOfElementLocated(hybridScreen))\n .click();\n\nMobileElement input = (MobileElement) wait\n .until(ExpectedConditions.presenceOfElementLocated(urlInput));\n\n// Get into the webview and assert that we're not yet at the correct page\nString webContext = getWebContext(driver);\ndriver.context(webContext);\nAssert.assertNotEquals(driver.getTitle(), title);\n\n// Go back into the native context and automate the URL button\ndriver.context(\"NATIVE_APP\");\ninput.sendKeys(\"https://google.com\");\nWebElement navigate = driver.findElement(navigateBtn);\nnavigate.click();\n\n// Assert that going to Google is not allowed\nThread.sleep(1000); // cheap way to ensure alert has time to show\ndriver.switchTo().alert().accept();\n\n// Now try to go to Appium Pro\ndriver.findElement(clearBtn).click();\ninput.sendKeys(\"https://appiumpro.com\");\nnavigate.click();\n\n// Go back into the webview and assert that the title is correct\ndriver.context(webContext);\nwait.until(ExpectedConditions.titleIs(title));\n```\n\nYou can see that we make use of our `getWebContext` helper method by storing a reference to the webview context. (Really, we should also have a null check and throw if a webview context is not available, because that means our test will fail! But for now we assume the webview is always present). Then, we go through the steps outlined above: asserting that the initial webview page's title is not the same as the Appium Pro site, then proving we can't navigate to Google, then finally directing the app to navigate to Appium Pro, and asserting that the title of the webview page now reflects that situation. You can see that in webview mode, we can even use Selenium-only client features, like the `titleIs` expected condition.\n\nThe code above is completely cross-platform; I can run this just the same on both the iOS and Android versions of the TheApp. Once you're in a webview, iOS- and Android-specific issues mostly disappear. How is this possible? There is an awful lot of machinery that we've put into Appium to make hybrid and web testing seamless. On iOS we hook into the Remote Debugger port exposed by every webview when the app is in debug mode, and leverage the Selenium Atoms to facilitate use of the Selenium API via injecting JS. On Android, we run Chromedriver under the hood as a subprocess, and attach it to the particular webview we care about---this works because webviews on Android are backed by Chrome.\n\nSpeaking of Chrome, one wrinkle of hybrid automation on Android is that each release of Chromedriver has a minimum version of Chrome which it supports. This means that if you get a version of Appium which bundles Chromedriver version X, and the version of Chrome on your device is older than the minimum Chrome version for X, webviews will not be automatable. If you run into that situation, refer to the [Chromedriver section of the Appium docs](http://appium.io/docs/en/writing-running-appium/web/chromedriver/) with instructions on using special flags to get versions of Chromedriver appropriate for your automation.\n\nThat's it for basic hybrid automation! There's a [full code sample](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition017_Hybrid_Apps.java) included below. In it you'll notice that there are no special capabilities being used to start the session---just the typical `app` capability. This is the main difference between web and hybrid automation with Appium: for web testing, you include the `browserName` capability and set it to `Safari` or `Chrome`, and you're automatically put into the web context on session start. For hybrid testing, you just use your own app reference as with native automation, and you're responsible for managing the contexts so Appium knows whether you're interested in native or web elements and actions.\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.MobileElement;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.ArrayList;\nimport javax.annotation.Nullable;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.WebElement;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition017_Hybrid_Apps {\n\n private String APP_IOS = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.5.0/TheApp-v1.5.0.app.zip\";\n private String APP_ANDROID = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.5.0/TheApp-v1.5.0.apk\";\n\n\n private static By hybridScreen = MobileBy.AccessibilityId(\"Webview Demo\");\n private static By urlInput = MobileBy.AccessibilityId(\"urlInput\");\n private static By navigateBtn = MobileBy.AccessibilityId(\"navigateBtn\");\n private static By clearBtn = MobileBy.AccessibilityId(\"clearBtn\");\n\n @Test\n public void testAppiumProSite_iOS() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.3\");\n capabilities.setCapability(\"deviceName\", \"iPhone 6\");\n capabilities.setCapability(\"app\", APP_IOS);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n\n @Test\n public void testAppiumProSite_Android() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", APP_ANDROID);\n\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n\n @Nullable\n private String getWebContext(AppiumDriver driver) {\n ArrayList contexts = new ArrayList(driver.getContextHandles());\n for (String context : contexts) {\n if (!context.equals(\"NATIVE_APP\")) {\n return context;\n }\n }\n return null;\n }\n\n public void actualTest(AppiumDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n final String title = \"Appium Pro: The Awesome Appium Tips Newsletter\";\n\n try {\n wait\n .until(ExpectedConditions.presenceOfElementLocated(hybridScreen))\n .click();\n\n MobileElement input = (MobileElement) wait\n .until(ExpectedConditions.presenceOfElementLocated(urlInput));\n\n // Get into the webview and assert that we're not yet at the correct page\n String webContext = getWebContext(driver);\n driver.context(webContext);\n Assert.assertNotEquals(driver.getTitle(), title);\n\n // Go back into the native context and automate the URL button\n driver.context(\"NATIVE_APP\");\n input.sendKeys(\"https://google.com\");\n WebElement navigate = driver.findElement(navigateBtn);\n navigate.click();\n\n // Assert that going to Google is not allowed\n Thread.sleep(1000); // cheap way to ensure alert has time to show\n driver.switchTo().alert().accept();\n\n // Now try to go to Appium Pro\n driver.findElement(clearBtn).click();\n input.sendKeys(\"https://appiumpro.com\");\n navigate.click();\n\n // Go back into the webview and assert that the title is correct\n driver.context(webContext);\n wait.until(ExpectedConditions.titleIs(title));\n } catch (InterruptedException ign) {\n } finally {\n driver.quit();\n }\n\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Automating Cross-Platform Hybrid Apps","shortDesc":"Hybrid apps contain both native and web components, melded together in some ratio to provide a single unified app experience for the user. With Appium and the various Context commands, it's possible to successfully automate both halves of hybrid apps across both iOS and Android.","liveAt":"2018-05-16 10:00","canonicalRef":"https://www.headspin.io/blog/automating-cross-platform-hybrid-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/17-automating-cross-platform-hybrid-apps","slug":"17-automating-cross-platform-hybrid-apps","path":"/editions/17-automating-cross-platform-hybrid-apps","news":null,"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0016.md","num":16,"rawMd":"\n\nCopying and pasting is one of those classic pairings, like peanut butter and jelly, or pizza and beer, or chips and guacamole. (Hmm, am I hungry?). For programmers, copying and pasting is bad---we should refactor our code instead. For users of mobile apps, copying and pasting is a wonderful way to save time typing and remembering. For this reason both iOS and Android platforms offer system-level clipboard capabilities, which apps can hook into to incorporate into their own functionality.\n\nIf you find yourself automating an app which has behavior tied to the clipboard, you might wonder how you can automate this behavior. As a first pass, you might try to do it the way a user would: by finding or typing some text, then long-pressing, then selecting the \"copy\" or \"paste\" context buttons which appear. Unfortunately, these buttons are \"special\" and don't show up in the automation hierarchy, which means this technique won't work reliably.\n\nThankfully, Appium now exposes getter and setter commands for the clipboard across both platforms. The specifics of the way the clipboard works, and what you can do with it, vary by platform (for example, setting image data is possible with the iOS clipboard). However, the basic flow of copying and pasting text works the same way across both platforms, and this is our topic for this article.\n\nTo showcase this type of automation, I've created [version 1.4.0 of The App](https://github.com/cloudgrey-io/the-app/releases/tag/v1.4.0), which has a new \"Clipboard Demo\" view:\n\n

\n \"Screenshot\n

\n\nBasically, we can either tap the \"Refresh\" button and get the contents of the clipboard displayed on screen, or we can enter our own text into a text field and set that text as the text content of the clipboard. We'll use this little demo view to understand Appium's clipboard methods. Our test steps will look like:\n\n1. Set the clipboard text to \"Hello World\" using Appium\n2. Tap the \"Refresh\" button\n3. Verify that the text displayed on the view is \"Hello World\"\n4. Enter \"Hello World Again\" into the text field\n5. Tap the \"Set\" button\n6. Get the clipboard text using Appium\n7. Verify the clipboard text was \"Hello World Again\"\n\nThese test steps prove that our app does exactly what it purports to do (which is admittedly pretty dumb).\n\nEagle-eyed testers will have noticed that the seven steps above actually constitute two tests, one that proves our app correctly displays the clipboard text and one that proves it correctly sets it. Let's take steps 1-3 first, and look at how we would write it up in an Appium script:\n\n```java\nString text = \"Hello World\";\ndriver.setClipboardText(text);\ndriver.findElement(refreshClipboardBtn).click();\nAssert.assertEquals(driver.findElement(clipboardText).getText(), text);\n```\n\n(Here we assume we've got some `By` locators all ready for us to use). The magic method is `driver.setClipboardText`, which enables us to set the clipboard text to anything we want. We're able to use this technique to prove beyond a shadow of a doubt that when we hit the \"Refresh\" button the displayed text actually came from the clipboard.\n\nAnd of course this method has a twin, which enables the second test (steps 4-7):\n\n```java\ntext = \"Hello World Again\";\ndriver.findElement(clipboardInput).sendKeys(text);\ndriver.findElement(setTextBtn).click();\nAssert.assertEquals(driver.getClipboardText(), text);\n```\n\nHere, all we're doing is entering a unique string into the text field, and hitting the \"Set Clipboard Text\" button. Then, we're using Appium's `getClipboardText` method to retrieve the clipboard text from the device, so we can verify it was actually set via the app. If the strings match, we've proved our app was able to set the clipboard text successfully.\n\nThat's all there is to it! This works in a fully cross-platform manner, as our full example will show, with tests for each of iOS and Android that both call out to the same `automateClipboard` method:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.clipboard.HasClipboard;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.junit.runner.RunWith;\nimport org.junit.runners.JUnit4;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\n@RunWith(JUnit4.class)\npublic class Edition016_Clipboard {\n\n private String ANDROID_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.4.0/TheApp-v1.4.0.apk\";\n private String IOS_APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.4.0/TheApp-v1.4.0.app.zip\";\n private String AVD_NAME = \"emu27\";\n private String APPIUM_SERVER = \"http://localhost:4723/wd/hub\";\n\n private By clipboardNav = MobileBy.AccessibilityId(\"Clipboard Demo\");\n private By refreshClipboardBtn = MobileBy.AccessibilityId(\"refreshClipboardText\");\n private By clipboardInput = MobileBy.AccessibilityId(\"messageInput\");\n private By setTextBtn = MobileBy.AccessibilityId(\"setClipboardText\");\n\n @Test\n public void testClipboard_Android() throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"avd\", AVD_NAME);\n capabilities.setCapability(\"app\", ANDROID_APP);\n\n AndroidDriver driver = new AndroidDriver(new URL(APPIUM_SERVER), capabilities);\n automateClipboard(driver);\n }\n\n @Test\n public void testClipboard_iOS() throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.3\");\n capabilities.setCapability(\"deviceName\", \"iPhone 7\");\n capabilities.setCapability(\"app\", IOS_APP);\n\n IOSDriver driver = new IOSDriver(new URL(APPIUM_SERVER), capabilities);\n automateClipboard(driver);\n }\n\n private void automateClipboard(AppiumDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 5);\n\n try {\n wait.until(ExpectedConditions.presenceOfElementLocated(clipboardNav)).click();\n\n String text = \"Hello World\";\n ((HasClipboard) driver).setClipboardText(text);\n wait.until(ExpectedConditions.presenceOfElementLocated(refreshClipboardBtn)).click();\n By clipboardText = MobileBy.AccessibilityId(text);\n Assert.assertEquals(driver.findElement(clipboardText).getText(), text);\n\n text = \"Hello World Again\";\n driver.findElement(clipboardInput).sendKeys(text);\n try {\n driver.hideKeyboard();\n } catch (Exception ign) {}\n driver.findElement(setTextBtn).click();\n Assert.assertEquals(((HasClipboard) driver).getClipboardText(), text);\n\n } finally {\n driver.quit();\n }\n }\n}\n```\n\nIn this example, we have two tests, one for iOS and one for Android, each of which is responsible solely for setting up its unique capabilities. Then, the heavy lifting is done by the `automateClipboard` method, which simply takes an `AppiumDriver` and works its magic. Because we don't have an instance of `IOSDriver` or `AndroidDriver` specifically, we have to cast our driver instance to `HasClipboard` in order to make sure we still have access to the clipboard methods (which properly belong only on the platform-specific drivers). And of course, in the full example, we use explicit waits to make sure we don't run into any timing issues with element presence.\n\nIf you fancy a deeper look at the code, you can always visit its home [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition016_Clipboard.java).\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Automating the Clipboard on iOS and Android","shortDesc":"Both iOS and Android allow copying and pasting of text and other types of content. Apps can hook into this native clipboard and provide custom experiences based on clipboard content. Appium gives you special commands to automate the clipboard across both platforms.","liveAt":"2018-05-09 10:00","canonicalRef":"https://www.headspin.io/blog/automating-the-clipboard-on-ios-and-android"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/16-automating-the-clipboard-on-ios-and-android","slug":"16-automating-the-clipboard-on-ios-and-android","path":"/editions/16-automating-the-clipboard-on-ios-and-android","news":{"rawMd":"This week everyone please welcome Appium Pro's newest sponsor, Applitools! Sponsors help keep the Appium Pro newsletter coming out every week, and I'm thrilled that Applitools is joining Sauce Labs in supporting this labor of love for the Appium community.\n","num":16,"mdPath":"/vercel/path0/content/news/0016.md"},"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0014.md","num":14,"rawMd":"\n\nThe first-ever [AppiumConf](https://appiumconf.com) was held recently (April 6 2018), and it was a landmark event for the Appium community, bringing together Appium users and developers from all over the world. It was full of interesting talks, so do check out [the full talk recordings](https://appiumconf.com/talks) or read my [conference recap](https://saucelabs.com/blog/appiumconf-2018-retrospective-and-highlights) to get a feel for the event.\n\nI had the privilege of closing out the day with a final session, and I wanted to do something special for this inaugural conference. So, I decided to showcase Appium's automation abilities, not for the purpose of testing, but as part of a provocative musical performance, merging my loves for code, music, and philosophy into one experience. If you're not sure what I'm talking about, just watch [the demo portion of my talk](https://youtu.be/29c22KqtG3s?t=1770) before reading further.\n\n

\n \"JLipps\n

\n\nIn this edition of Appium Pro, I thought we would have some fun together, go behind the scenes of my demo, and answer the question, \"How do you automate a pop rock band with Appium?\"\n\n(Apologies in advance to those of you who are used to seeing Java for the code snippets; this week they will be in JS, since that's my language of choice for hacking, and the language I wrote my demo in).\n\n### Step 1: Write a Song\n\nIt goes without saying that no band can do anything without a song, whether it's a band of human beings or software instruments played by software robots. To write a song, it helps to know something about music, so that might be the place for some of you to start, namely learning to play an instrument or reading up on some music theory. It's not absolutely necessary to add a vocal melody and lyrics, but it's a pretty core piece of most pop music, so it doesn't hurt to engage your brain's right hemisphere and get poetic.\n\n(I certainly encourage lyrics that explore the unforeseen tribulations of a post-human techno-reality, ironically layered on top of positive ukulele-driven pop-rock, but of course you're not limited to that. In fact, now that I think about it, I don't know if there's room enough in the ironic-post-human-techno-critic genre for the both of us!)\n\nIf you've never written a song before, it's helpful to break it down into its constituent parts, just like approaching any other problem. Focus on one thing at a time, and layer ideas and instruments in harmonious ways until the whole is greater than the sum of the parts. Most pop songs have 2-4 themes (by \"theme\" I mean chord sequences and melody lines that recur throughout the song). If you can come up with a verse, a chorus, and a bridge, for example, you're all set. Make the chorus the most catchy and interesting part. Just go back and forth between these themes in some order or another for 3-4 minutes and you're good to go. Oh, and you'll need a title. I already picked \"Ghost in the Machine\", so you can't have it!\n\nAt the end of this step, you'll have a nice arrangement of your tune, ready to be played by all the instruments you want to include (for example, I chose ukulele, voice, electric guitar, bass, and drums, but you don't need that many). I also chose to record the song with real instruments at this point, because I liked it enough I wasn't going to be satisfied with Appium's shoddy timekeeping, and I wanted a fun party favor to give the conference attendees. (It's OK, Appium Pro readers, [you can listen to it too](https://jlipps.com/ghost)).\n\n

\n \"Screenshot\n

\n\n### Step 2: Find (or Make) Your Mobile Instruments\n\nNow, since you want Appium to play most or all of your instruments, you need to find mobile apps which give you the ability to make the appropriate sounds. These are all over the place, however they're mostly of limited quality. I really liked one called \"Souvey Musical Pro\" but I can't seem to find it on the play store nowadays. For the bass part, I ended up using [ORG 2018](https://play.google.com/store/apps/details?id=com.sofeh.android.musicstudio3) on a Samsung S7.\n\n

\n \"Screenshot\n

\n\nFor drums, I decided to play around with the WebAudio API and wrote a little sampler app. A sampler simply provides buttons that, when pressed, play pre-recorded sounds (the \"samples\"). Most of the apps of the kind I mentioned above are actually samplers (though some provide synthesized MIDI-like sounds instead, and sound worse). I called my little app [WebSampler](https://github.com/jlipps/appium-orchestra/tree/master/lib/websampler) (many thanks to [this HTML5 drum pad app](https://demo.agektmr.com/drumpad/) for giving me some code to inspect and start hacking on), and it's basically a tiny bit of HTML, CSS, and JS that together enable the displaying of arbitrary buttons on the screen, linked to arbitrary `.wav` files. For the drums, I used some free samples I found online of a Pearl kit, and decided to run them via an iPhone simulator.\n\n

\n \"Screenshot\n

\n\nI then searched in vain for a way to play decent-sounding electric guitar parts using a mobile app. This is very difficult, because a guitar is physically set up very differently than keyboards, which are much more suited to being tapped on a screen. Unlike the simple bass guitar part I wrote, my electric guitar part required many notes to sound at the same time, and for pretty quick transitions between the notes. At the end of the day, I decided quality of sound was more important for my demo than faithful modeling of the playing of each note. So, I extended the WebSampler app to include a library of electric guitar samples, which I recorded from my own guitar using the [Apogee Duet](http://www.apogeedigital.com/products/duet/product-tour) (which, incidentally, I cannot recommend more highly; it blows every other portable DAC out of the water). Essentially, each sonically-unique measure of the guitar part got sliced into a sample, and then given its own button in the WebSampler screen:\n\n

\n \"Screenshot\n

\n\nThat accounted for all the instruments I wanted my robot band to automate, because I myself was going to sing and play the ukulele. I did, however, realize that my words would have a much bigger impact if they could be seen and not just heard, so I whipped up another little app I called [WebWriter](https://github.com/jlipps/appium-orchestra/tree/master/lib/webwriter). Actually it's not even an app, it's just an HTML page with some basic styling and a single text box, whose sole purpose is to have Appium (or Selenium, as it turned out to be) write words into the box.\n\nAt this point, you have everything you need to make music. In fact, if you had some musically-inclined friends and a typist around, you could perform the digital version of your song now, with your friends tapping the sampler buttons or virtual keyboards. But our goal today is robot-driven rock, so we must press on.\n\n### Step 3: Teach Appium to Play the Mobile Instruments\n\nNow we get to the core nugget of how Appium is involved in this whole picture. We need a way to automate the tapping and UI navigating that's required to make our mobile instruments work. Luckily, that's Appium's whole job. So for each instrument, create a class (or some other abstraction) which encapsulates the Appium commands necessary to:\n\n1. Get the app into the right state for playing the instrument (sometimes it's necessary to navigate through some menus, adjust the octave, etc...)\n2. Play a particular note or sound by tapping (either on an element or on a coordinate)\n\nAs an example of what #2 could look like, here's a code snippet from my [ORG2018 instrument class](https://github.com/jlipps/appium-orchestra/blob/master/instruments/org2018.js):\n\n```javascript\nasync playNote (note, durationSecs) {\n let pos = this.posForNote(note);\n await this.tapPos(pos.x, pos.y, this.getRealDuration(durationSecs));\n}\n\nasync playChord (chord, durationSecs) {\n const positions = chord.map(note => this.posForNote(note));\n await this.multiPos(positions, this.getRealDuration(durationSecs));\n}\n```\n\nBasically, these are the core methods which take a note and a duration and turn it into actual taps on the device. In the case of this instrument, actual labeled elements were not available in the UI hierarchy, and so I had to construct a map of notes to x/y coordinates, which works based on percentage width of each key on the screen. You can see we have the option to play either a single note or a chord, using inherited methods called `tapPos` and `multiPos` (inherited from a [base `Instrument` class](https://github.com/jlipps/appium-orchestra/blob/master/lib/instrument.js)). These methods are slightly more complicated, but it's essentially just plain old Appium code:\n\n```javascript\nasync tapPos (x, y, duration = null) {\n let action = new wd.TouchAction();\n if (duration) {\n action.longPress({x, y, duration: parseInt(duration * 1000, 10)});\n action.release();\n } else {\n action.tap({x, y});\n }\n await this.driver.performTouchAction(action);\n}\n\nasync multiPos (positions, duration = null) {\n let actions = [];\n let multi = new wd.MultiAction();\n duration = parseInt((duration || 0) * 1000, 10);\n for (let pos of positions) {\n let action = new wd.TouchAction();\n action.longPress({x: pos.x, y: pos.y, duration});\n action.release();\n actions.push(action);\n }\n multi.add(...actions);\n await this.driver.performMultiAction(multi);\n}\n```\n\nThis is the core link between the world of Appium and the world of music. Everything else, as they say, is a simple matter of programming.\n\n### Step 4: Teach Appium to Read Music\n\nJust because we have a nice way for Appium to hit some notes on a mobile instrument app doesn't mean it's a musician. To get to the next level, we have to teach it to read music. If we wanted, we could invent some crazy computer vision-based method for taking actual music notation (the kind humans read) and turning it into the `playNote` method above. This, however, is way too much work.\n\nInstead, we'll devise a new kind of music notation which is much easier to write up as code. Basically, all we care about is ensuring that Appium hits the right note (or notes) at the right time. We don't care about dynamics, time signatures, or any of the other things which feature in actual music notation. So, what we can do is simply specify a sequences of notes and note durations in one long string. Take this string, for example:\n\n
\nc4 d8 e8 f8 g8 a8 b8 C4 r2.\n
\n\nThe letters correspond to musical notes of the same name, in this case representing an ascending C Major scale. Most instruments can play several or many versions of the same note, placed at different octaves (multiples of a sonic frequency), but in ASCII we have only two ways of denoting the same letter: lowercase and uppercase. This means that our musical notation will be limited to 2 octaves (which is a practical limitation of most instrument apps anyway; octave changes happen outside the keyboard by tapping a button, for example). There's also a special letter \"r\" (\"r\" is not a musical note), which stands for \"rest\", meaning silence.\n\nThe numbers correspond to note lengths. Here we invert actual musical nomenclature, however: \"4\" means a \"quarter\" note (or one quarter of a full 4/4 measure), not \"4 beats\", because writing \"1/4\" is too clunky. \"8\" means an \"eighth\" note, \"2\" means a half note, and \"1\" means a whole note. A dot (\".\") after the number means to add an extra half of the specified duration. So `c4.` would signify a \"dotted quarter note\", or a note lasting for the duration of a quarter note plus an eighth note.\n\nFinally, if we want to play a chord instead of a single note, we simply use the \"/\" character in between the notes that comprise the chord (as shown in this classic IV-I Plagal cadence):\n\n
\nf/a/C2 e/g/C2\n
\n\nSince, at the end of the day all we care about is notes and durations, a string with the specifications above is sufficient! Now all that's necessary is some code that can parse these string representations and turn them into calls to the `playNote` and `playChord` methods we saw above. In my own demo, for a variety of reasons, I separated this into two logically distinct components: first, a [method](https://github.com/jlipps/appium-orchestra/blob/master/lib/instrument.js#L87-L149) that takes a tempo and a score string, analyzes the score, throws any errors, asserts appropriate length of the score based on other information, and finally returns a sequence of notes with their temporal offset already calculated. I pulled out this functionality because I wanted to run it in advance of the song, to avoid wasting any previous note-playing time trying to calculate or parse stuff.\n\nSecond, there is [another method](https://github.com/jlipps/appium-orchestra/blob/master/lib/instrument.js#L151-L188) which takes the note sequence as input and simply calls the instrument `play` methods at the appropriate time for each note.\n\nThis is, in fact, how we get Appium to stick to the correct tempo. Each note it plays has a pre-calculated offset from the start time of the song. So, after playing each note, Appium can simply calculate how much time it should wait until it plays the next note. If it sees that it is running behind for whatever reason, it can drop notes and simply pick up with the next one it has a chance of hitting on time! Thus, as long as each instrument's part is kicked off at the same time, they can run completely independently of each other and still automatically correct for any lag, giving the illusion of keeping tempo with one another.\n\n(In this section I focused on music notation, but let's not forget about our lyric writer \"instrument\"---what sort of notation do we use for writing lyrics in time with the rest of the instruments? You can see that notation [here](https://github.com/jlipps/appium-orchestra/blob/master/songs/ghost-in-the-machine/lyrics.js).)\n\n### Step 5: Teach Appium to Play Your Parts\n\nNow that we have a nice simple spec for playing music in general, we have to commit _our_ song to code. This is perhaps the most tedious part of the whole process, not least because our musical notation is very error-prone. Each part, even though it consists of thousands of separate notes across hundreds of different measures, needs to end up with _exactly_ the same number of beats. If we'd wanted, we could have introduced the concept of time signatures and perhaps made it easier to cope with this kind of difficulty. But, since this is just a demo and not an app we need to maintain in production, we can just use good old-fashioned brainpower to carry us through.\n\nEssentially, we want to end up with one score file for each part, each of which exports its score string. My song, for example, had a [bass part](https://github.com/jlipps/appium-orchestra/blob/master/songs/ghost-in-the-machine/score-bass.js) with this code as a portion of its score file:\n\n\n```js\nconst LEADIN = ['r1 r1'];\nconst INTRO = ['r1 '.repeat(8)];\nconst VERSE = [\n 'b4. b2 r8 ', 'f#4. C#2 r8 ',\n 'e4. b2 r8 ', 'b4. b2 r8 ',\n 'g#4. D#2 r8 ', 'f#4. b2 r8 ',\n 'f#4. C#2 r8 ', 'f#4. C#2 r8 ',\n];\n\nconst BRIDGE_PART = [\n 'E2 E4 F#4', 'C#2 C#2',\n 'a#2 a#4 C#4', 'b2 b2'\n];\n```\n\nYou can see here that I've separated measures as different strings for the benefit of my own sanity. And at the end of the file, I take all the different parts of the song and concatenate them back together into one big score string:\n\n```js\nconst SONG = [].concat(\n LEADIN,\n INTRO,\n VERSE,\n VERSE,\n BRIDGE,\n CHORUS,\n TAG,\n BREAK,\n VERSE,\n BRIDGE,\n CHORUS,\n TAG_2,\n OUTRO,\n).join(\" \");\n\nmodule.exports = SONG;\n```\n\nI named the different parts of the song because there's a fair amount of repetition and it's nice to reuse the note segments rather then rewrite them. It's also nice to see the structure of your song laid out this way. It shows how simple and how repetitive it really is. If anything, I would say my song had a more complex structure than most pop songs, so hopefully yours is even a bit simpler.\n\n(You can also take a look at [all the parts](https://github.com/jlipps/appium-orchestra/tree/master/songs/ghost-in-the-machine) for my song, including drums, guitar, and lyrics).\n\n### Step 6: Set Up Your Orchestra\n\nThe last bit of code we'll need to write is the bit which sets up all the various instruments and their respective drivers, triggers their startup routines, and finally kicks off the song for each of them. I've called this the [`Director`](https://github.com/jlipps/appium-orchestra/blob/master/lib/director.js), and it has a pretty simple job. And of course, when we hand off all the musicians to the director, we have to set up all the appropriate variables, like ports of the running Appium servers, and so on, so our code knows which Appium server is connected to which instrument, and so on.\n\nWhat we end up with is the final piece of code, called [song.js](https://github.com/jlipps/appium-orchestra/blob/master/songs/ghost-in-the-machine/song.js), which we can run to kick off our song. It's basically full of constant-setting boilerplate, apart from the important bit:\n\n```js\nlet d = new Director(TEMPO, true);\ndrums.analysis = d.addPart(drums, SCORE_DRUMS);\neg.analysis = d.addPart(eg, SCORE_ELECS);\nbass.analysis = d.addPart(bass, SCORE_BASS);\nd.addLyrics(lyrics, LYRICS);\ntry {\n await d.assemble();\n await d.play();\n} finally {\n await d.dismiss();\n}\n```\n\nHere we take our instrument objects, pass them into the Director, and then ask the director to play our song! Of course we wrap the song in a `try` block, because if something goes wrong (which can sometimes happen, unfortunately), we want to clean up our sessions.\n\n### Step 7: Perform Your Heart Out\n\nThe next and final step is something only you can do. Get up on a stage somewhere, kick off the Appium band, and sing and play your song with all the human passion and little imperfections that make for a truly engaging performance. And don't forget to let me know when you do, so I can check it out!\n\nSpeaking of little imperfections, I realized after I watched the video of my talk that there was a fairly serious bug in Appium's performance. At the last minute I decided to manually rotate the iOS simulator to landscape mode, however I inadvertently did so after the drum pad elements had already been found and their coordinates cached. The result was that for the entire song, the wrong drum part was being played! Luckily, it was still being played in keeping with the tempo, so it wasn't obvious to a casual listener (or even me, in the moment). A good reminder that something interesting and unforeseen is bound to happen in any live performance.\n\nAnyway, if you've followed this tutorial, and added your own signature flair, what you'll have ended up with is something possibly very similar to the overall architecture that I've described in this article, which could be shown in a diagram like this:\n\n

\n \"Architectural\n

\n\nWe've now reached the end of our little ratiocination on doing something extra crazy with Appium. Music is just one of many avenues, of course. As I told the attendees of my talk, we should all be looking for interesting and supposedly \"frivolous\" things to do with the technology available to us. If we only use our technology for boring and utilitarian purposes, we'll run the risk of becoming technologized ourselves. Don't forget to let the creativity out once and a while! I'll leave you now with the lyrics to my song, [Ghost in the Machine](https://jlipps.com/ghost):\n\n
\n[VERSE]\nI believed the things I heard\nThose that said I'm better served\nScanning my whole brain\nAnd going digital\n\n[VERSE]\nSo now that man is dead and gone\nAnd here I am still humming along\nBut I can't shake the feeling that there's something wrong\n\n[BRIDGE]\nAnd feeling was the first to go\nI never thought I'd stoop so low\nBut now I want my body back\nIts limits seem a welcome lack\nI'm hailed a virtual deity\nBut still no good at being me\n\n[CHORUS]\nI am the ghost in the machine\nGod of everything I see\nBut in my new reality\nEyes are a rare commodity\nI don't just want to be a brain\nI want to be a human being\nBut my mind's stuck in silicon\nAnd I'm stuck inside my mind\n\n[TAG]\nI am the perfect simulation of\nThe not-so-perfect person that I was (x2)\nSo won't you get me out of here\n\n[VERSE]\nI review the memories\nThat I have saved of sun and trees\nBut one thing that I cannot do\nIs feel them like they're mine\n
\n\n_(Many thanks to [Spareroom Studios](http://spareroomstudios.com) in San Francisco for drum composition, general production, mixing, and mastering of the recorded version of the song)_\n","metadata":{"lang":"JS","platform":"All Platforms","subPlatform":"All Devices","title":"How to Automate a Pop Rock Band","shortDesc":"Sometimes you need a mobile software robot like Appium for something other than your day job. In this article I break down my AppiumConf 2018 demo, giving a behind-the-scenes tour of how I managed to coax Appium to act as the director of a software band which accompanied my performance of the original song 'Ghost in the Machine'.","liveAt":"2018-04-25 10:00","canonicalRef":"https://www.headspin.io/blog/how-to-automate-a-pop-rock-band"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/14-how-to-automate-a-pop-rock-band","slug":"14-how-to-automate-a-pop-rock-band","path":"/editions/14-how-to-automate-a-pop-rock-band","news":null,"tags":["JS","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0013.md","num":13,"rawMd":"\n\nSmartphone users use on average 9 apps per day, and 30 apps per month, according to [one recent report](https://techcrunch.com/2017/05/04/report-smartphone-owners-are-using-9-apps-per-day-30-per-month/). If your app is lucky enough to be given such attention, what are the chances that it lives alone, with no integration to the rest of the user's smartphone experience? So many apps today provide value by integrating with other aspects of the mobile phone experience, even if it's just as simple as taking photos using the device's camera.\n\nHistorically, testing these kinds of multi-app integrations with iOS has been challenging, because Apple's automation model allowed automation only so long as the AUT was running. The second you switched to another app to automate another part of your user flow, the automation would die. Thankfully, with Appium's XCUITest driver (available with iOS 9.3+), we're no longer limited to this kind of sandbox.\n\nThere are so many kinds of multi-app flows we could experiment with, but let's imagine a scenario involving the camera. Let's say your app allows the user to take photos, and modifies them in some way before finally saving them to the system-wide photo library (the Camera Roll). One key verification for testing your app will be ensuring that, after performing the necessary steps in _your_ app, the photo is newly available in the Camera Roll album of the _Photos_ app. So, the test would look like this on a high level:\n\n1. Launch the Photos app\n2. Navigate to Camera Roll album\n3. Determine how many photos are present, and store that number\n4. Launch your app\n5. Add and save photo by automating your app's UI\n6. Switch back to the Photos app\n7. Navigate to Camera Roll\n8. Determine how many photos are present now\n9. Verify that the number of photos present before has increased by 1\n\nThis is all possible to encode into your Appium scripts using two handy commands: `mobile: launchApp` and `mobile: activateApp`. We've seen `launchApp` before, in the [Appium Pro article on testing app upgrades](https://appiumpro.com/editions/6). `activateApp` is just the same, only the app must already have been launched; `activateApp` merely brings it back to the foreground. Here are examples of how we'd use these commands, by incorporating the Bundle IDs of the apps we're dealing with (having these Bundle IDs available is a pre-requisite):\n```java\n// launch the photos app (with the special bundle id seen below)\nHashMap args = new HashMap<>();\nargs.put(\"bundleId\", \"com.apple.mobileslideshow\");\ndriver.executeScript(\"mobile: launchApp\", args);\n\n// re-activate that AUT (in this case The App)\nargs.put(\"bundleId\", \"io.cloudgrey.the-app\");\ndriver.executeScript(\"mobile: activateApp\", args);\n```\n\nSimple, no? With Appium we can really pretend we are a user and do all the things a user would do in order to check the appropriate conditions. I won't go into detail in this edition about actually navigating the Photos app UI, nor does The App currently allow you to take photos and save them to the library, so filling out these specific steps will be left as an exercise to the reader. And chances are, you have a different requirement than verifying photo saving. So the thing to take away is the principle of using `mobile: launchApp` [and friends](https://appium.io/docs/en/writing-running-appium/ios/ios-xctest-mobile-apps-management/index.html) to manage apps on the iOS device during a test.\n\nWhat's nice is that this particular strategy will work not only for simulators but also for real devices! So if you need to have a user login to your app via Facebook, or some other third-party credential authority, you can do so.\n\nAs always, there's a [full example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition013_iOS_Multi_App.java) up on GitHub. In this case, though, we just insert some clever `Thread.sleep`s instead of doing all the craziness with finding the number of photos and so on. Check it out here:\n\n```java\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition013_iOS_Multi_App {\n\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.3.0/TheApp-v1.3.0.app.zip\";\n private String PHOTOS_BUNDLE_ID = \"com.apple.mobileslideshow\";\n private String BUNDLE_ID = \"io.cloudgrey.the-app\";\n\n @Test\n public void testMultiApps() throws Exception {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.3\");\n capabilities.setCapability(\"deviceName\", \"iPhone X\");\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n try {\n HashMap args = new HashMap<>();\n args.put(\"bundleId\", PHOTOS_BUNDLE_ID);\n driver.executeScript(\"mobile: launchApp\", args);\n\n // Here is where we would navigate the Photos UI to get the number of Camera Roll photos\n Thread.sleep(1000);\n\n // Now reactivate our AUT\n args.put(\"bundleId\", BUNDLE_ID);\n driver.executeScript(\"mobile: activateApp\", args);\n\n // Here is where we would cause the new photo to be taken, edited, and saved\n Thread.sleep(1000);\n\n // Now reactivate the Photos app\n args.put(\"bundleId\", PHOTOS_BUNDLE_ID);\n driver.executeScript(\"mobile: activateApp\", args);\n\n // Here is where we would get the new number of Camera Roll photos, and assert that it\n // has increased by the appropriate number\n Thread.sleep(1000);\n } finally {\n driver.quit();\n }\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Switching Between iOS Apps During a Test","shortDesc":"Sometimes being able to automate your app-under-test (AUT) is not enough. People don't have just one app on their phone, and oftentimes your app depends on actions taken in another app. How do you test multi-app flows like this? With Appium, of course!","liveAt":"2018-04-18 10:00","canonicalRef":"https://www.headspin.io/blog/switching-between-ios-apps-during-a-test "},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/13-switching-between-ios-apps-during-a-test","slug":"13-switching-between-ios-apps-during-a-test","path":"/editions/13-switching-between-ios-apps-during-a-test","news":{"rawMd":"Did you know that we just wrapped up the first ever [Appium Pro Workshop](https://appiumpro.com/workshops) in San Francisco? Check out [my Twitter feed](https://twitter.com/jlipps) for some fun pictures from it. Stay tuned for lots more workshops this year! And don't forget we still have a few spots open in our [London workshop](https://appiumpro.com/workshops) in March.\n","num":13,"mdPath":"/vercel/path0/content/news/0013.md"},"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0012.md","num":12,"rawMd":"\n\nAs mobile app testing becomes more and more ubiquitous, the lines between different kinds of automated testing can become blurred. For example, performance testing is becoming an integral part of the development cycle. And for good reason---users of mobile apps have very low tolerance for poorly performing apps. We've already shown how it's possible to [use Appium to collect performance metrics for Android apps](https://appiumpro.com/editions/5), and we can do something very similar for iOS.\n\nPerformance testing for iOS apps involves the [Instruments](https://developer.apple.com/library/content/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/index.html) utility distributed by Apple alongside Xcode. Instruments comes with a number of built-in analyses and measurements. If you open it up, you're greeted with a list of these:\n\n

\n \"Instruments\n

\n\nEssentially, these are the various performance measurements it will be possible to initiate using Appium, so keep an eye on this list for ideas about what you might want to measure for your app, and make sure to check out Apple's docs if you want to know more about what each of these do. For our purposes in this newsletter, we're going to choose the \"Time Profiler\" instrument. But since we're using Appium, we don't need to click on anything in the Instruments app itself. Instead, we'll head to our code editor!\n\nThe way iOS profiling works with Appium is with two commands: one to start the profiling and one to stop it and dump the data out to us for viewing. These commands are available as of Appium 1.8, via the `mobile:` command interface. Essentially, it's as simple as:\n\n```java\ndriver.executeScript(\"mobile: startPerfRecord\", args);\n\n// here: do some stuff in the app that you want to profile\n\nString b64Zip = (String)driver.executeScript(\"mobile: stopPerfRecord\", args);\n\n// here: convert the base64-encoded zip file to actual file data on your system\n```\n\nWe use the `mobile: startPerfRecord` and `mobile: stopPerfRecord` commands to signal to Appium when during our script we'd like the profiling to occur. There is one wrinkle, however: for any of this to work, we need to have started the Appium server with the `--relaxed-security` flag. This is because Instruments can gather data from the system as a whole, not just the AUT. (It's thus a security risk to expose potentially sensitive system information to Appium sessions running from a remote client, for example in the context of a cloud Appium host).\n\nThere's also another aspect of the snippet above that I haven't yet touched on: what about the `args` parameter to these methods? The \"start\" method takes an argument object with three fields, for example:\n\n```java\nHashMap args = new HashMap<>();\nargs.put(\"pid\", \"current\");\nargs.put(\"profileName\", \"Time Profiler\");\nargs.put(\"timeout\", 60000);\n```\n\nHere we have:\n\n1. Specified which process we want to attach to (\"current\" is a handy shortcut to refer to the AUT, which is probably what we're interested in. By default _all_ processes will be profiled if we don't specify anything).\n2. Specified which kind of instrument we want to run (the Time Profiler).\n3. Specified a timeout (in milliseconds) after which the performance trace will stop on its own. These trace files can get pretty huge so this is an important parameter to remember.\n\nFor `stopPerfRecord`, the only argument we care about is `profileName`, which should have the same value as what we passed in to `startPerfRecord`, so Appium knows which of potentially multiple traces to stop. The other wrinkle here is the return value of `stopPerfRecord`; what's `b64Zip` supposed to mean? Well, what Appium is giving back to you when you stop performance recording is actually an [Instruments Trace Document](https://developer.apple.com/library/content/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/WorkingwithProfileDocuments.html), which happens to be a directory under the hood. Since directories are impossible to send in string format, Appium zips up this `.trace` directory and hands it back to the client script in base64 encoding. To make use of this data, we have to decode it and dump it into a zipfile on our system, with code like the following:\n\n```java\nFile traceZip = new File(\"/path/to/trace.zip\");\nString b64Zip = (String)driver.executeScript(\"mobile: stopPerfRecord\", args);\nbyte[] bytesZip = Base64.getMimeDecoder().decode(b64Zip);\nFileOutputStream stream = new FileOutputStream(traceZip);\nstream.write(bytesZip);\n```\n\nAt this point, we'll have a nice little `trace.zip` sitting at the specified location on disk. We can now simply unzip it and double-click it to open the trace file up in the Instruments viewer:\n\n

\n \"Instruments\n

\n\nIn this Instruments UI, we can dig through the various threads that were active during the profiled portion of our Appium test, and see which routines that thread spent most of its time in (via the stacktrace snapshots taken by the profiler). This can help us to find CPU-hungry areas of our app, which we might decide to offload to a worker thread to improve the user experience, for example. There are all kinds of considerations, and potential avenues of improvement based on the data gleaned from these trace files, but that is outside the scope of this brief tutorial. What's important today is that you've figured out how to capture the data!\n\nSince a common use case might be to profile your app over time, you might consider attaching the zipped trace files to your test report in your CI system, so that if a test fails, you also have some juicy profile data that could help in remediating the test. (There's actually an easy way to send the zip file straight to an asset manager that supports HTTP uploads; check out the [Appium XCUITest performance docs](https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/ios/ios-xctest-performance.md) for more info).\n\nFor the sake of showing a full example, the following is a simple test which lifts the actual app behavior from a different edition of Appium Pro, and simply runs those steps a number of times while bracketed by performance recording. The zip file is then written to disk, just as above, where I can happily open up the report in Instruments.\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.File;\nimport java.io.FileOutputStream;\nimport java.net.URL;\nimport java.util.Base64;\nimport java.util.HashMap;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition012_iOS_Performance {\n\n\n private String APP = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.3.0/TheApp-v1.3.0.app.zip\";\n private File traceZip;\n\n private By msgInput = By.xpath(\"//XCUIElementTypeTextField[@name=\\\"messageInput\\\"]\");\n private By savedMsg = MobileBy.AccessibilityId(\"savedMessage\");\n private By saveMsgBtn = MobileBy.AccessibilityId(\"messageSaveBtn\");\n private By echoBox = MobileBy.AccessibilityId(\"Echo Box\");\n\n private String TEST_MESSAGE = \"Hello World\";\n\n @Test\n public void testAppActivity() throws Exception {\n\n // Note: Appium server must have been started with --relaxed-security\n\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.3\");\n capabilities.setCapability(\"deviceName\", \"iPhone X\");\n capabilities.setCapability(\"app\", APP);\n\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n traceZip = new File(\"/path/to/trace.zip\");\n\n try {\n HashMap args = new HashMap<>();\n args.put(\"timeout\", 60000);\n args.put(\"pid\", \"current\");\n args.put(\"profileName\", \"Time Profiler\");\n driver.executeScript(\"mobile: startPerfRecord\", args);\n\n performActions(driver);\n performActions(driver);\n performActions(driver);\n\n args = new HashMap<>();\n args.put(\"profileName\", \"Time Profiler\");\n String b64Zip = (String)driver.executeScript(\"mobile: stopPerfRecord\", args);\n byte[] bytesZip = Base64.getMimeDecoder().decode(b64Zip);\n FileOutputStream stream = new FileOutputStream(traceZip);\n stream.write(bytesZip);\n } finally {\n driver.quit();\n }\n }\n\n public void performActions(IOSDriver driver) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n wait.until(ExpectedConditions.presenceOfElementLocated(echoBox)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(msgInput)).sendKeys(TEST_MESSAGE);\n wait.until(ExpectedConditions.presenceOfElementLocated(saveMsgBtn)).click();\n String savedText = wait.until(ExpectedConditions.presenceOfElementLocated(savedMsg)).getText();\n Assert.assertEquals(savedText, TEST_MESSAGE);\n driver.navigate().back();\n }\n}\n```\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Capturing Performance Data for Native iOS Apps","shortDesc":"App usability is about more than functionality: if the user experience isn't snappy enough, it can be a big problem. Performance testing and analysis is an important phase of testing. Thankfully, Appium can gather performance data during the course of your iOS tests.","liveAt":"2018-04-11 10:00","canonicalRef":"https://www.headspin.io/blog/capturing-performance-data-for-native-ios-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/12-capturing-performance-data-for-native-ios-apps","slug":"12-capturing-performance-data-for-native-ios-apps","path":"/editions/12-capturing-performance-data-for-native-ios-apps","news":{"rawMd":"AppiumConf happened last Friday! And what a day it was. The videos should be released soon, and I'll be sure to highlight some interesting bits from the conference in future newsletters. I also got a lot of great ideas for newsletter topics, so even if you weren't able to make it to AppiumConf yourself, you'll benefit from the insight of the many wonderful speakers, and other interesting conversations I had with Appium users there. Stay tuned!\n","num":12,"mdPath":"/vercel/path0/content/news/0012.md"},"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0010.md","num":10,"rawMd":"\n\nWhen the Appium server runs there is often a bewildering stream of logs, giving\ninformation about the automation in process, but the details are often obscure.\n\n### Server Startup\nThe first lines of the log announce the version of Appium, and the address it is running on:\n```\n$ appium\n[Appium] Welcome to Appium v1.8.0-beta3 (REV 40e40975ebd3593d08c3f83de2546258f7ddf11d)\n[Appium] Appium REST http interface listener started on 0.0.0.0:4723\n```\nFurther, if you have started the server with any [special flags](http://appium.io/docs/en/writing-running-appium/server-args/),\nincluding [default desired capabilities](http://appium.io/docs/en/writing-running-appium/default-capabilities-arg/)\nthey will also be noted here, to help with understand the environment that is being run:\n```\n$ appium --address 172.19.131.113 --port 8000 --default-capabilities '{\"showIOSLog\": true}'\n[Appium] Welcome to Appium v1.8.0-beta3 (REV 40e40975ebd3593d08c3f83de2546258f7ddf11d)\n[Appium] Non-default server args:\n[Appium] address: 172.19.131.113\n[Appium] port: 8000\n[Appium] defaultCapabilities: {\n[Appium] showIOSLog: true\n[Appium] }\n[Appium] Default capabilities, which will be added to each request unless overridden by desired capabilities:\n[Appium] showIOSLog: true\n[Appium] Appium REST http interface listener started on 172.19.131.113:8000\n```\nThis information is invaluable for providing context to the test automation that will be run. In particular, different\nversions of Appium will have different capacities and issues, so knowing what version is running is necessary\nto begin to make any determination of what is going on in any run.\n\n### Session Creation\nWhile session creation involves a complicated array of operations to get the\nenvironment set up and the application under test running, the beginning of the\nlogs for a command to create the session provides basic information about the\nsession. In particular, the [desired capabilities](http://appium.io/docs/en/writing-running-appium/caps/),\nalong with any [default capabilities](http://appium.io/docs/en/writing-running-appium/default-capabilities-arg/)\nare listed. It is often useful to check that what was intended to be requested was\nactually received by the Appium server, since it is the capabilities listed here\nthat will be acted upon for the automation session.\n```\n[Appium] Creating new XCUITestDriver (v2.68.0) session\n[Appium] Capabilities:\n[Appium] app: /Users/isaac/apps/UICatalog-iphonesimulator.app\n[Appium] platformName: iOS\n[Appium] platformVersion: 11.3\n[Appium] deviceName: iPhone 6\n[Appium] automationName: XCUITest\n[Appium] noReset: true\n[Appium] maxTypingFrequency: 30\n[Appium] clearSystemFiles: true\n[Appium] showXcodeLog: false\n[debug] [BaseDriver]\n[debug] [BaseDriver] Creating session with MJSONWP desired capabilities: {\"app\":\"/Users/isaac/code/a...\n```\n\n### Appium Commands\nAppium is a [REST](https://en.wikipedia.org/wiki/Representational_state_transfer)\nserver, accepting incoming HTTP requests, performing the action requested, and returning a result\nof some sort. In the Appium server logs, each such incoming request is delineated by a line indicating the request, and\na line indicating the response. In between these are the details of the execution of the requested command:\n```\n[HTTP] --> GET /wd/hub/status {}\n[debug] [MJSONWP] Calling AppiumDriver.getStatus() with args: []\n[debug] [MJSONWP] Responding to client with driver.getStatus() result: {\"build\":{\"version\":\"1.8.0-beta3\",\"revision\":\"30e7b45bdc5668124af33c41492aa5195fcdf64d\"}}\n[HTTP] <-- GET /wd/hub/status 200 121 ms - 126\n```\n\n### Investigating Errors\nOn the client side errors are usually helpful, there is usually more information\nto be found in the logs. Usually the errors will be at the end of the automation\nsession, but sometimes the session can continue and the error logs will be earlier.\nSo the first step is to identify the command where the error happened. As we have\nalready seen, each command is marked by `[HTTP] -->` and `[HTTP] <--`. Within these\nmarkers is the details of the execution of the command, including any error output.\n\nLet's take a look at a concrete example:\n\n```\n[HTTP] --> POST /wd/hub/session\n\n[debug] [AndroidDriver] Shutting down Android driver\n[debug] [AndroidDriver] Called deleteSession but bootstrap wasn't active\n[debug] [Logcat] Stopping logcat capture\n[debug] [ADB] Getting connected devices...\n[debug] [ADB] 1 device(s) connected\n[debug] [ADB] Running '/home/user/Android/Sdk/platform-tools//adb' with args: [\"-P\",5037,\"-s\",\"ec8c4df\",\"shell\",\"am\",\"force-stop\",\"io.appium.unlock\"]\n[debug] [AndroidDriver] Not cleaning generated files. Add `clearSystemFiles` capability if wanted.\n[MJSONWP] Encountered internal error running command: Error: Cannot stop and clear com.company.app. Original error: Error executing adbExec. Original error: 'Command '/home/user/Android/Sdk/platform-tools//adb -P 5037 -s ec8c4df shell pm clear com.company.app' exited with code 1'; Stderr: 'Error: java.lang.SecurityException: PID 22126 does not have permission android.permission.CLEAR_APP_USER_DATA to clear data of package com.company.app'; Code: '1'\n at Object.wrappedLogger.errorAndThrow (../../lib/logging.js:63:13)\n at ADB.callee$0$0$ (../../../lib/tools/adb-commands.js:334:9)\n at tryCatch (/home/linuxbrew/.linuxbrew/lib/node_modules/appium/node_modules/babel-runtime/regenerator/runtime.js:67:40)\n at GeneratorFunctionPrototype.invoke [as _invoke] (/home/linuxbrew/.linuxbrew/lib/node_modules/appium/node_modules/babel-runtime/regenerator/runtime.js:315:22)\n at GeneratorFunctionPrototype.prototype.(anonymous function) [as throw] (/home/linuxbrew/.linuxbrew/lib/node_modules/appium/node_modules/babel-runtime/regenerator/runtime.js:100:21)\n at GeneratorFunctionPrototype.invoke (/home/linuxbrew/.linuxbrew/lib/node_modules/appium/node_modules/babel-runtime/regenerator/runtime.js:136:37)\n at \n at process._tickCallback (internal/process/next_tick.js:188:7)\n[HTTP] <-- POST /wd/hub/session 500 40811 ms - 557\n```\n\nIn this abridged set of logs, the user has attempted to start a session using the Android driver, and has been met with an error. In this case it looks like the error happened during Appium's attempt to stop and clear the AUT in preparation for the session. Here the error gives us two important pieces of information:\n\n1. What Appium was trying to do, specifically\n2. What went wrong (in the form of error text and any related codes)\n\nIn this case, Appium was trying to run an `adb` command (`adb shell am force-stop`), with parameters that have been included in the error text. And what happened? It was met with an Android system error about permissions. At this point, we have everything we need to troubleshoot locally. We can run the `adb` command that Appium tried to run, for example, and determine whether we can reproduce the issue outside of Appium. If so, we can then seek recourse through a helpful Android-specific forum, or from other Appium users who might have run into the same problem. If the command can be run successfully from the command line, then it may be we have uncovered a bug in Appium, and it should be reported as a [GitHub issue](https://github.com/appium/appium/issues).\n\n(If you're curious, in this case the user was able to reproduce the problem outside of Appium, so it appeared to be related to the specific device manufacturer's security model).\n\nObviously, this is one of many, many examples that could have been given, but it illustrates the core point that, when faced with an error, the logs can help provide more information that facilitate a local repro attempt, or help the Appium team understand what could be going wrong. No issue should ever be submitted to the Appium issue tracker without a complete set of logs!\n\n### Server Flags for Changing the Logging Output\nWhile the default logging output is often enough, and if you are opening an\n[issue on Github](https://github.com/appium/appium/issues) to get help with a\nproblem, the more information the better, there are [server flags](http://appium.io/docs/en/writing-running-appium/server-args/)\nprovided to change the logging behavior of the Appium server.\n\n* `--log-level` - change the level at which Appium logs information. Appium defaults\n to logging _everything_, which can be a lot. The options for the flag are\n `'info'`, `'info:debug'`, `'info:info'`, `'info:warn'`, `'info:error'`,\n `'warn'`, `'warn:debug'`, `'warn:info'`, `'warn:warn'`, `'warn:error'`,\n `'error'`, `'error:debug'`, `'error:info'`, `'error:warn'`, `'error:error'`,\n `'debug'`, `'debug:debug'`, `'debug:info'`, `'debug:warn'`, `'debug:error'`\n* `--log-no-colors` - If your console does not display colors (this will be manifested\n as logs with odd character sequences like `TODO: find the color`) you can turn\n off colors with this flag\n* `--log-timestamp` - Add a timestamp to the beginning of each log line, which is\n useful in cases where timeouts are occurring. The log lines will look like\n ```\n 2018-03-15 13:17:58:663 - [Appium] Welcome to Appium v1.8.0-beta3 (REV 30e7b45bdc5668124af33c41492aa5195fcdf64d)\n 2018-03-15 13:17:58:664 - [Appium] Non-default server args:\n 2018-03-15 13:17:58:665 - [Appium] logTimestamp: true\n 2018-03-15 13:17:58:732 - [Appium] Appium REST http interface listener started on 0.0.0.0:4723\n ```\n","metadata":{"lang":"All Languages","platform":"All Platforms","subPlatform":"All Devices","title":"Anatomy of Logging in Appium","shortDesc":"Running into problems with Appium can be a frustrating experience. Sometimes you just get a cryptic, unhelpful message as part of an error thrown in your client code. Where do you go next? The Appium logs! This is a guest post from Isaac Murchie on how to take advantage of reading the Appium logs to help understand what's going on and potentially resolve issues.","introMsg":"This is a guest post from Isaac Murchie, Head of Open Source at Sauce labs and longtime Appium maintainer. He'll be presenting at AppiumConf in a few weeks on the topic of how to profitably read your Appium logs to help resolve issues. This is a sneak preview of what he'll be discussing at the conference. Enjoy! --Jonathan","liveAt":"2018-03-21 10:00","canonicalRef":"https://www.headspin.io/blog/anatomy-of-logging-in-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/10-anatomy-of-logging-in-appium","slug":"10-anatomy-of-logging-in-appium","path":"/editions/10-anatomy-of-logging-in-appium","news":null,"tags":["All Languages","All Platforms","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0009.md","num":9,"rawMd":"\n\nPreviously, we explored how to [test app upgrades on iOS](https://appiumpro.com/editions/6), and this week we're going to do the same for Android. What's the deal with app upgrades? The idea is that as testers, our responsibility goes beyond the functionality of an app in just one version. We also care about issues that might arise as users migrate from one version to another. Your customers are *upgrading* to the latest version, not installing it from scratch. There might be all kinds of existing user data that needs to be migrated for the app to work correctly, and simply running functional tests of the new version would not be sufficient to catch these cases.\n\nAppium comes with some built-in commands to handle this kind of requirement. Fortunately, on Android it's even simpler than on iOS! To demonstrate, let's go back to the old version of [The App](https://github.com/cloudgrey-io/the-app), specifically [v1.0.0](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.0). This version contains just one feature: a little text field which saves what you write into it and echoes it back to you. It just so happens that this data is saved internally using the key `@TheApp:savedEcho`. (The specifics of how this data is saved have to do with React Native and aren't important---just imagine that your dev team has a way to save local user data that might need to persist between upgrades).\n\nNow, imagine that some crazy developer has decided that this key just absolutely needs to change to `@TheApp:savedAwesomeText`. What this means is that if a user were to simply upgrade to the new version of the app after having saved some text in the old version of the app, the app would find no saved text under the new storage key! In other words, the upgrade would break the user's experience of the app. In this case, the fix is to provide some migration code in the app itself which looks for data at the old key and moves it over to the new key if any is there.\n\nThat's all the responsibility of the app developer (in this case, me). Let's pretend that like a good developer I initially forgot to write the migration code after changing the data storage key. I then release [v1.0.1](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.1). This version will unfortunately contain the bug I described above (of the missing text) due to the forgotten migration code---even though as a standalone version of the app it works fine. Eventually I realize the mistake, and write the migration code. I can't re-release v1.0.1, so I release the fix as [v1.0.2](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.2).\n\nAt this point, the testers, having been burned by my incompetence as a developer, want to write an automated test to prove that upgrading to v1.0.2 will actually work (and of course, are themselves facepalming that they didn't write such an upgrade test before v1.0.1 was released!).\n\nSo what do we need in terms of Appium code? Basically, we take advantage of these two methods:\n\n```java\ndriver.installApp(\"/path/to/apk\");\n\n// 'activity' is an instance of import io.appium.java_client.android.Activity;\ndriver.startActivity(activity);\n```\n\nThis set of commands:\n\n1. Replaces the existing app with the one given in the path (or URL) as the argument to `installApp`, and in so doing stops the old version of the app from running (so there is no need to explicitly stop the app before installing the new one).\n2. Starts up the new version of the app using its package name and whatever activity you choose.\n\nThe only wrinkle here is that the argument to `startActivity` must be of type `Activity`. There is a separate `Activity` class because there are a number of options that can be passed to this method. We're only interested in the basic ones for the purposes of app upgrades, so we can simply create our `Activity` object as follows:\n\n```java\nActivity activity = new Activity(\"com.mycompany.myapp\", \"MyActivity\");\ndriver.startActivity(activity);\n```\n\nFor our example, to create a passing app upgrade test we'll of course need two apps: our original version and the version to upgrade to. In our case, that's v1.0.0 and v1.0.2 of The App:\n\n```js\nprivate String APP_V1_0_0 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.0/TheApp-v1.0.0.apk\";\nprivate String APP_V1_0_2 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.2/TheApp-v1.0.2.apk\";\n```\n\nI'm happily using GitHub asset download URLs to feed into Appium here. Assuming we've started the test with `APP_V1_0_0` as our `app` capability, the duo of app upgrade commands then looks like:\n\n```java\ndriver.installApp(appUpgradeVersion);\n\nActivity activity = new Activity(APP_PKG, APP_ACT);\ndriver.startActivity(activity);\n```\n\nFor the full test flow, we want to:\n\n1. Open up v1.0.0\n2. Add our saved message and verify it is displayed\n3. Upgrade to v1.0.2\n4. Open the app and verify that the message is still displayed -- this proves that the old message has been migrated to the new key\n\nHere's the code for the implementation of this full flow, including boilerplate:\n\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.Activity;\nimport io.appium.java_client.android.AndroidDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.junit.runner.RunWith;\nimport org.junit.runners.JUnit4;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\n@RunWith(JUnit4.class)\npublic class Edition009_Android_Upgrade {\n\n private String APP_PKG = \"io.cloudgrey.the_app\";\n private String APP_ACT = \"com.theapp.MainActivity\";\n\n private String APP_V1_0_0 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.0/TheApp-v1.0.0.apk\";\n private String APP_V1_0_1 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.1/TheApp-v1.0.1.apk\";\n private String APP_V1_0_2 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.2/TheApp-v1.0.2.apk\";\n\n\n private String TEST_MESSAGE = \"Hello World\";\n\n private By msgInput = MobileBy.AccessibilityId(\"messageInput\");\n private By savedMsg = MobileBy.AccessibilityId(TEST_MESSAGE);\n private By saveMsgBtn = MobileBy.AccessibilityId(\"messageSaveBtn\");\n private By echoBox = MobileBy.AccessibilityId(\"Echo Box\");\n\n @Test\n public void testSavedTextAfterUpgrade () throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", APP_V1_0_0);\n\n // change this to APP_V1_0_1 to experience a failing scenario\n String appUpgradeVersion = APP_V1_0_2;\n\n // Open the app.\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n wait.until(ExpectedConditions.presenceOfElementLocated(echoBox)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(msgInput)).sendKeys(TEST_MESSAGE);\n wait.until(ExpectedConditions.presenceOfElementLocated(saveMsgBtn)).click();\n String savedText = wait.until(ExpectedConditions.presenceOfElementLocated(savedMsg)).getText();\n Assert.assertEquals(savedText, TEST_MESSAGE);\n\n driver.installApp(appUpgradeVersion);\n Activity activity = new Activity(APP_PKG, APP_ACT);\n driver.startActivity(activity);\n\n wait.until(ExpectedConditions.presenceOfElementLocated(echoBox)).click();\n savedText = wait.until(ExpectedConditions.presenceOfElementLocated(savedMsg)).getText();\n Assert.assertEquals(savedText, TEST_MESSAGE);\n } finally {\n driver.quit();\n }\n }\n}\n```\n\n(Note that I included the option of running the test with the version of the app that has the bug, namely v1.0.1, which would produce a failing test.)\n\nAs always, you can find the code inside a working repository [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition009_Android_Upgrade.java). That's it for testing app upgrades on Android! Remember to check out the previous tip on [how to do the same thing with iOS](https://appiumpro.com/editions/6).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Testing Android App Upgrades","shortDesc":"It's important to test that user data and user experiences are not broken due to changes between app version upgrades. Appium comes with some built-in app management commands that make it possible to test how your app behaves across versions, all within the context of one Appium session.","liveAt":"2018-03-14 10:00","canonicalRef":"https://www.headspin.io/blog/testing-android-app-upgrades"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/9-testing-android-app-upgrades","slug":"9-testing-android-app-upgrades","path":"/editions/9-testing-android-app-upgrades","news":{"rawMd":"If you're reading this newsletter just after it landed in your inbox, there's still time to sign up for the [AppiumConf Panel Webinar](http://info.saucelabs.com/Appium-Panel-Webinar-LP-A.html) (today at 10AM PDT / 1PM EDT / 5PM GMT). A bunch of us who will be participating in AppiumConf are going to be having an informal chat. Come and check it out (or watch it after the fact)!\n","num":9,"mdPath":"/vercel/path0/content/news/0009.md"},"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0008.md","num":8,"rawMd":"\n\nFinding elements for use in your Appium tests is done by means of one of a number of *locator strategies*. Appium inherited a number of locator strategies from Selenium, and different Appium drivers have also added some new locator strategies to make finding elements faster and more effective. Let's take a look at all the locator strategies available, and which ones are supported in the current flagship iOS driver (the [XCUITest driver](https://appium.io/docs/en/drivers/ios-xcuitest/)):\n\n| Locator Strategy | From | Available? |\n| ----------------------- | --------------- | --------------------------------- |\n| `class name` | Selenium | Yes |\n| `id` | Selenium | Yes |\n| `name` | Selenium | Yes |\n| `xpath` | Selenium | Yes |\n| `accessibility id` | Appium | Yes |\n| `-ios predicate string` | Appium | Yes |\n| `-ios class chain` | Appium | Yes |\n| `css selector` | Selenium | No |\n| `link text` | Selenium | No |\n| `partial link text` | Selenium | No |\n| `tag name` | Selenium | No |\n| `-ios uiautomation` | Appium | No |\n| `-android uiautomator` | Appium | No |\n\nSo, for our iOS tests we have access to 7 locator strategies, including the infamous [XPath](https://en.wikipedia.org/wiki/XPath). Before we look at alternatives, let's explore why using the XPath locator strategy is attractive, and why it is often a bad idea regardless.\n\n### Why (Not) XPath?\n\n XPath is attractive because it is a hierarchy-based locator strategy, and can thus find any element in the DOM (for Selenium) or app hierarchy (for Appium). Often, apps come to us in such a form that elements are not annotated as they should be for the sake of testability. It's easy to rely on XPath to find an element with a selector like `//*[1]/*[1]/*[3]/*[2]/*[1]/*[1]`, especially when there's seemingly no other way to find that element.\n\nOf course, relying on a selector like this is a horrible idea, because it will be invalidated by just about any change to your app hierarchy. That's the trouble with relying on a hierarchical description of your element---the app hierarchy is typically far from stable. It could change between runs (because different data is displayed, for example), or between versions of your app (because the app devs add a new component, or simply change an underlying UI library).\n\nWith the XCUITest driver, it can also be a horrible idea because XPath might be very slow. XPath and its cousin, XML, are not native languages for iOS development. In order to provide access to elements via XPath, the Appium team has had to perform a bunch of technical magic. Essentially, every time you run an XPath query, the entire app hierarchy must be recursively walked and serialized into XML. This by itself can take a lot of time, especially if your app has many elements. Then, once the XPath query has been run on this XML serialization, the list of matching elements must be deserialized into actual element objects before their WebDriver representations are returned to your client call.\n\nIn sum, XPath is a mixed bag. Even if you avoid typical XPath pitfalls by using better, more restrictive, selectors (like `//XCUIElementTypeButton[@name=\"Foo\"]`), you will still incur the cost of generating the XML document to begin with. If you've encountered slowness with XPath and the XCUITest driver, it's probably because your app has so many elements it's revealing this core inefficiency I just described. What's the solution? Don't use XPath!\n\nOf course, if you can get a direct handle on your element using the `id`, `name`, or `accessibility id` strategies, that is to be preferred above all else. Stop reading---you're done! But if you are in a position where there is no unique id or label associated with your element, and XPath has turned out to be too slow, consider using the `-ios predicate string` or `-ios class chain` locator strategies.\n\n### iOS Predicate String Strategy\n\n[Predicate Format Strings](https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/Predicates/Articles/pSyntax.html) are a typical Apple dev thing, and they also work in iOS. Predicate format strings enable basic comparisons and matching. In our case, they allow basic matching of elements according to simple criteria. What's really useful about predicate strings is that you can combine simple criteria to form more complex matches. In the XCUITest driver, predicate strings can be used to match various element attributes, including `name`, `value`, `label`, `type`, `visible`, etc...\n\nOne example from the [WebDriverAgent predicate string guide](https://github.com/facebook/WebDriverAgent/wiki/Predicate-Queries-Construction-Rules) shows a fun compound predicate:\n\n```bash\ntype == 'XCUIElementTypeButton' AND value BEGINSWITH[c] 'bla' AND visible == 1\n```\n\nThis predicate string would match any visible button whose value begins with 'bla'. How would we write this up in our Java client code? Simply by using the appropriate `MobileBy` method, as follows:\n\n```java\nString selector = \"type == 'XCUIElementTypeButton' AND value BEGINSWITH[c] 'bla' AND visible == 1\";\ndriver.findElement(MobileBy.iOSNsPredicateString(selector));\n```\n\nBecause predicate matching is [built into XCUITest](https://developer.apple.com/documentation/xctest/xcuielementquery/1500768-element), it has the potential to be much faster than Appium's XPath strategy.\n\n### iOS Class Chain Strategy\n\nThe final option is a sort of hybrid between XPath and predicate strings: the `-ios class chain` locator strategy. This was developed by the Appium team to meet the need of hierarchical queries in a more performant way. The types of queries possible via the class chain strategy are not as powerful as those enabled by XPath, but this restriction means a better performance guarantee (this is because it is possible to map class chain queries into a series of direct XCUITest calls, rather than having to recursively build an entire UI tree). Class chain queries look very much like XPath queries, however the only allowed filters are basic child/descendant indexing or predicate string matching. It's worth checking out the [class chain docs](https://github.com/facebook/WebDriverAgent/wiki/Class-Chain-Queries-Construction-Rules) to find a number of examples. Let's take a look at just a couple:\n\n* `XCUIElementTypeWindow[2]` selects the second window in the hierarchy.\n* ```XCUIElementTypeWindow[`label BEGINSWITH \"foo\"`][-1]``` selects the last window whose label begins with `foo`.\n* ```**/XCUIElementTypeCell[`name BEGINSWITH \"C\"`]/XCUIElementTypeButton[10]``` selects the 10th child button of the first cell in the tree whose name starts with `C` and which has at least ten direct children of type XCUIElementTypeButton.\n\nJust as before, there is a special `MobileBy` method to hook up your class chain queries in the Java client:\n\n```java\ndriver.findElement(MobileBy.iOSClassChain(selector));\n```\n\nAs you can tell from these examples, the class chain strategy allows for some quite powerful combinations of search filters. Next time you're experiencing slowness with XPath, reach for this strategy instead and see if you can express your query in its terms!\n\n### Caveats\n\nThere's no such thing as a free lunch, of course (even if you work in startups!). The main downside to these locator strategies is that they are not cross-platform. You have to know how the particular element attributes show up in iOS-land to use predicate strings (for example, `label` is an iOS-only concept). However, with appropriate separation of concerns in a page object model or equivalent, you might appreciate the performance gain of switching to one of these platform-specific locators. It's up to you!\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"How to Find Elements in iOS (Not) By XPath","shortDesc":"If you've used Appium for any length of time, you've probably encountered the sad fact that, sometimes, xpath element queries are slow. In this newsletter, we explore why that is the case for iOS, and explore the various alternatives to XPath available to us, especially the little-known 'class chain' locator strategy.","liveAt":"2018-03-07 06:00","canonicalRef":"https://www.headspin.io/blog/how-to-find-elements-in-ios-not-by-xpath"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/8-how-to-find-elements-in-ios-not-by-xpath","slug":"8-how-to-find-elements-in-ios-not-by-xpath","path":"/editions/8-how-to-find-elements-in-ios-not-by-xpath","news":{"rawMd":"This last week I attended [SauceCon 2018](https://saucecon.com). It was an extraordinary conference and lived up to my high expectations based on SauceCon 2017 last June. I talked to a bunch of people who were using Appium and Selenium and had an awesome time. (Though I probably shouldn't have stayed up super late that one night doing karaoke with the likes of [Dave Haeffner](https://twitter.com/TourDeDave), [Jim Evans](https://twitter.com/jimevansmusic), and [Marcus Merrell](https://twitter.com/mmerrell), among other hooligans).\n\nNow that the talk videos are online, here are some suggestions for you Appium fans:\n\n* [Lessons from a Decade in Selenium](https://www.youtube.com/watch?v=uHnJsh7Wyok&list=PL67l1VPxOnT4B8ClvBQXPwIV5LwQomF2o&index=2) --- Simon Stewart's keynote on lessons learned from hacking open source.\n* [The Death of Flaky Tests](https://www.youtube.com/watch?v=W-P56jq1AWA&list=PL67l1VPxOnT4B8ClvBQXPwIV5LwQomF2o&index=7) --- Dave Haeffner's thoughts on how to mitigate flakiness, and what flakiness even means to begin with.\n* [Transitioning from Selenium to Appium, How Hard Can it Be?](https://www.youtube.com/watch?v=OnSxkLlzXOQ&index=17&list=PL67l1VPxOnT4B8ClvBQXPwIV5LwQomF2o) --- Sergio Neves Barros on what it was like to try to use Appium in the early days, and war stories from getting mobile Safari working on real devices.\n* [Appium: How to Write a Symphony](https://www.youtube.com/watch?v=G2OuCaJiZ8g&list=PL67l1VPxOnT4B8ClvBQXPwIV5LwQomF2o&index=20) --- Dan Cuellar (original creator of Appium) dreaming about what's after WebDriver.\n* [How to Test Your Terminator](https://www.youtube.com/watch?v=PXRSlbv1Grc&t=0s&list=PL67l1VPxOnT5UMXMojduH_cMuBmlYl90K&index=6) --- Jason Huggins also dreaming about what's after WebDriver, with more robots and a birthday surprise.\n* [Drivers of Change: Appium's Hidden Engines](https://www.youtube.com/watch?v=EhZuuZ1uEZk&t=16s&list=PL67l1VPxOnT5UMXMojduH_cMuBmlYl90K&index=13) --- My own talk, in which I build an Appium driver on stage.\n\nThese were all very interesting to hear in person so I highly recommend the videos! Speaking of conferences, don't forget to register for [AppiumConf](https://appiumconf.com) --- it's coming up soon!\n","num":8,"mdPath":"/vercel/path0/content/news/0008.md"},"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0007.md","num":7,"rawMd":"\n\nOne of the biggest difficulties involved in functional testing is that functional tests can be _slow_. Some functional testing frameworks are faster than others, and I'll be the first to admit that Appium isn't always as speedy as I would like. Speed itself is a topic for another time! This edition is all about how to make automation speed less important overall, by building shortcuts into your testing.\n\nWhat do I mean by shortcuts? Let's take logging in to an app as an example. Most apps require some kind of login step. Before your tests can access all the functionality they're supposed to exercise, they must first log in. And if you are keeping your tests atomic, as you should, _every_ test has to log in separately. This is a huge waste of time. You only need to run Appium commands on your login screen when you are actively testing the login functionality. For every other test, it'd be great to just land somewhere behind the authentication barrier and get directly to business.\n\nWe can accomplish this on iOS and Android with something called a deep link. Deep links are special URLs that can be associated with specific apps. The developer of the app registers a unique URL scheme with the OS, and from then on, the app will come alive and will be able to do whatever it wants based on the content of the URL.\n\nWhat we need, then, is some kind of special URL that our app interprets as a login attempt. When it detects that it has been activated with that URL, it will perform the login behind the scenes and redirect the app to the correct logged-in view.\n\nI recently spent some time adding deep link interpretation into [The App](https://github.com/cloudgrey-io/the-app). Exactly how I did that is beyond the scope of this edition, and is specific to React Native apps in any case. There are plenty of tutorials online for adding deep link support to your iOS or Android app. But what I ended up with in [v1.2.1](https://github.com/cloudgrey-io/the-app/releases/tag/v1.2.1) is support for a deep link that looks like this:\n\n```\ntheapp://login//\n```\n\nIn other words, with The App installed on a device, we can direct the device to navigate to a URL with that form, and The App will wake up and perform authentication of the requested username/password combination behind the scenes, dumping the user to the logged-in area instantly. Great! Now how do we do the same thing with Appium?\n\nIt's easy: just use `driver.get`. Since the Appium client libraries just extend the Selenium ones, we can use the same command that Selenium would use to navigate to a browser URL. In our case, the concept of URL is just ... bigger. So let's say we want to log in with the username `darlene` and the password `testing123`. All we have to do is construct a test that opens a session with our App Under Test, and runs the following command:\n\n```java\ndriver.get(\"theapp://login/darlene/testing123\");\n```\n\nOf course, it's up to the app developer to build the ability to respond to URLs and perform the appropriate action with them. And you probably want to make sure that URLs used for testing are not turned on in production versions of your app! But it's worth building them into your app for testing, or convincing your developer to. Being able to set up arbitrary test state is the best way to cut off significant portions of time off your testing. In my own experimentation with The App, I noticed a savings of 5-10s on each test just by bypassing the login screen in this way. These savings add up!\n\nHere is a [full example](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition007_Deep_Linking.java). It shows two ways of testing that the logged-in experience works correctly: first by navigating the login UI with Appium, and then by using the deep linking trick described in this edition. It also shows an example of what an extremely simple inline Page Object Model might look like, as a way to reuse code as much as possible. You'll also notice that because my app is cross-platform, I'm able to run these tests on both iOS and Android with minimal code difference.\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport org.junit.Test;\nimport org.junit.runner.RunWith;\nimport org.junit.runners.JUnit4;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\n@RunWith(JUnit4.class)\npublic class Edition007_Deep_Linking {\n\n private String APP_IOS = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.2.1/TheApp-v1.2.1.app.zip\";\n private String APP_ANDROID = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.2.1/TheApp-v1.2.1.apk\";\n\n private String AUTH_USER = \"alice\";\n private String AUTH_PASS = \"mypassword\";\n\n @Test\n public void testLoginSlowIOS() throws IOException {\n IOSModel model = new IOSModel();\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), model.caps);\n runStepByStepTest(driver, model);\n }\n\n @Test\n public void testLoginSlowAndroid() throws IOException {\n AndroidModel model = new AndroidModel();\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), model.caps);\n runStepByStepTest(driver, model);\n }\n\n private void runStepByStepTest(AppiumDriver driver, Model model) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n wait.until(ExpectedConditions.presenceOfElementLocated(model.loginScreen)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(model.username)).sendKeys(AUTH_USER);\n wait.until(ExpectedConditions.presenceOfElementLocated(model.password)).sendKeys(AUTH_PASS);\n wait.until(ExpectedConditions.presenceOfElementLocated(model.loginBtn)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(model.getLoggedInBy(AUTH_USER)));\n } finally {\n driver.quit();\n }\n }\n\n @Test\n public void testDeepLinkForDirectNavIOS () throws IOException {\n IOSModel model = new IOSModel();\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), model.caps);\n runDeepLinkTest(driver, model);\n }\n\n @Test\n public void testDeepLinkForDirectNavAndroid () throws IOException {\n AndroidModel model = new AndroidModel();\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), model.caps);\n runDeepLinkTest(driver, model);\n }\n\n private void runDeepLinkTest(AppiumDriver driver, Model model) {\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n driver.get(\"theapp://login/\" + AUTH_USER + \"/\" + AUTH_PASS);\n wait.until(ExpectedConditions.presenceOfElementLocated(model.getLoggedInBy(AUTH_USER)));\n } finally {\n driver.quit();\n }\n }\n\n private abstract class Model {\n public By loginScreen = MobileBy.AccessibilityId(\"Login Screen\");\n public By loginBtn = MobileBy.AccessibilityId(\"loginBtn\");\n public By username;\n public By password;\n\n public DesiredCapabilities caps;\n\n abstract By getLoggedInBy(String username);\n }\n\n private class IOSModel extends Model {\n IOSModel() {\n username = By.xpath(\"//XCUIElementTypeTextField[@name=\\\"username\\\"]\");\n password = By.xpath(\"//XCUIElementTypeSecureTextField[@name=\\\"password\\\"]\");\n\n caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"iOS\");\n caps.setCapability(\"deviceName\", \"iPhone 7\");\n caps.setCapability(\"platformVersion\", \"11.2\");\n caps.setCapability(\"app\", APP_IOS);\n }\n\n public By getLoggedInBy(String username) {\n return MobileBy.AccessibilityId(\"You are logged in as \" + username);\n }\n }\n\n private class AndroidModel extends Model {\n AndroidModel() {\n username = MobileBy.AccessibilityId(\"username\");\n password = MobileBy.AccessibilityId(\"password\");\n\n caps = new DesiredCapabilities();\n caps.setCapability(\"platformName\", \"Android\");\n caps.setCapability(\"deviceName\", \"Android Emulator\");\n caps.setCapability(\"app\", APP_ANDROID);\n caps.setCapability(\"automationName\", \"UiAutomator2\");\n }\n\n public By getLoggedInBy(String username) {\n return By.xpath(\"//android.widget.TextView[@text=\\\"You are logged in as \" + username + \"\\\"]\");\n }\n }\n}\n```\n\n\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Speeding Up Tests With Deep Links","shortDesc":"Appium tests are full-UI functional tests. They can take a long time to run. It's a good idea to look for shortcuts so that your tests don't have to spend a ton of time setting up state, and can instead focus on just the bare minimum they need to validate. With a little work, you can accomplish this using deep links in both iOS and Android.","liveAt":"2018-02-28 06:00","canonicalRef":"https://www.headspin.io/blog/speeding-up-tests-with-deep-links"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/7-speeding-up-tests-with-deep-links","slug":"7-speeding-up-tests-with-deep-links","path":"/editions/7-speeding-up-tests-with-deep-links","news":{"rawMd":"I've been getting a few questions about the language used in these editions. At the moment I only have the bandwidth to write an edition for one language at a time. I chose Java because of its popularity among Appium users. I have been collecting programming language data from all of you who have signed up to this newsletter, and will prioritize which languages to add to the rotation based on that data. Even if Java isn't your favorite language, I encourage you to keep reading. Most of these tips are not related to Java per se, and with a bit of imagination or docs perusal, you'll be able to accomplish the same things in any Appium client library.\n","num":7,"mdPath":"/vercel/path0/content/news/0007.md"},"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0006.md","num":6,"rawMd":"\n\nThis week we wander back to the fun world of iOS to talk about testing app upgrades. One common requirement for many testers goes beyond the functionality of an app in a frozen state. Hopefully, your team is shipping new versions of your app to customers on a frequent basis. This means your customers are *upgrading* to the latest version, not installing it from scratch. There might all kinds of existing user data that needs to be migrated for the app to work correctly, and simply running functional tests of the new version would not be sufficient to catch these cases.\n\nLuckily, the latest version of the Appium 1.8 beta now supports commands that make it easy to reinstall and relaunch your app from within a single Appium session. This opens up the possibility of testing an in-place upgrade.\n\nTo demonstrate this, I'd like to introduce [The App](https://github.com/cloudgrey-io/the-app), a cross-platform React Native demo app that I will be building out to support the various examples we'll encounter here in Appium Pro. If the name sounds vaguely familiar, it might be because I'm infringing on my friend Dave Haeffner's trademark use of the definite article! (Dave is the author of [Elemental Selenium](http://elementalselenium.com), and hosts a demo web app called [The Internet](https://the-internet.herokuapp.com/)). Luckily for me, Dave has been gracious in donating the name idea to Appium Pro, so we can proceed without worry.\n\nI recently published [v1.0.0](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.0) of The App, and it contains just one feature: a little text field which saves what you write into it and echoes it back to you. It just so happens that this data is saved internally using the key `@TheApp:savedEcho`.\n\nNow, in our toy app world, imagine that some crazy developer has decided that this key just absolutely needs to change to `@TheApp:savedAwesomeText`. What this means is that if a user were to simply upgrade to the new version of the app after having saved some text in the old version of the app, the app would find no saved text under the new storage key! In other words, the upgrade would break the user's experience of the app. In this case, the fix is to provide some migration code in the app itself which looks for data at the old key and moves it over to the new key if any is there.\n\nThat's all the responsibility of the app developer (i.e., me). Let's pretend that instead of writing this migration code, I forgot, and went ahead and changed the key. I then released [v1.0.1](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.1). This version contains the bug of the missing text due to the forgotten migration code, even though as a standalone version of the app it works fine. Let's keep pretending, and say that after much facepalming I wrote the migration code and released it as [v1.0.2](https://github.com/cloudgrey-io/the-app/releases/tag/v1.0.2). Now the intrepid testers who are rightly suspicious of me want to write an automated test to prove that upgrading to v1.0.2 will actually work (and of course, are themselves facepalming that they didn't write such an upgrade test before v1.0.1 was released!).\n\nSo what do we need in terms of Appium code? Basically, we take advantage of these three `mobile:` methods:\n\n```java\ndriver.executeScript(\"mobile: terminateApp\", args);\n\ndriver.executeScript(\"mobile: installApp\", args);\n\ndriver.executeScript(\"mobile: launchApp\", args);\n```\n\nThis set of commands stops the current app (which we need to do explicitly so that the underlying WebDriverAgent instance doesn't think the app has crashed on us), installs the new app over top of it, and then launches the new app.\n\nWhat do the `args` look like? In each case they should be a `HashMap` that is used to generate simple JSON structures. The `terminateApp` and `launchApp` commands both have one key, `bundleId` (which is of course the bundle ID of your app). `installApp` takes an `app` key, which is the path or URL to the new version of your app.\n\nFor our example, to create a passing test we'll need two apps: v1.0.0 and v1.0.2:\n\n```js\nprivate String APP_V1_0_0 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.0/TheApp-v1.0.0.app.zip\";\nprivate String APP_V1_0_2 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.2/TheApp-v1.0.2.app.zip\";\n```\n\nI'm happily using GitHub asset download URLs to feed into Appium here. Assuming we've started the test with `APP_V1_0_0` as our `app` capability, the trio of app upgrade commands then looks like:\n\n```java\nHashMap bundleArgs = new HashMap<>();\nbundleArgs.put(\"bundleId\", BUNDLE_ID);\ndriver.executeScript(\"mobile: terminateApp\", bundleArgs);\n\nHashMap installArgs = new HashMap<>();\ninstallArgs.put(\"app\", APP_V1_0_2);\ndriver.executeScript(\"mobile: installApp\", installArgs);\n\n// can just reuse args for terminateApp\ndriver.executeScript(\"mobile: launchApp\", bundleArgs);\n```\n\nFor the full test flow, we want to:\n\n1. Open up v1.0.0\n2. Add our saved message and verify it is displayed\n3. Upgrade to v1.0.2\n4. Open the app and verify that the message is still displayed -- this proves that the old message has been migrated to the new key\n\nHere's the code (note that I included the option of running the test with the version of the app that has the bug, which would produce a failing test):\n\n```java\nimport io.appium.java_client.MobileBy;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.io.IOException;\nimport java.net.URL;\nimport java.util.HashMap;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.junit.runner.RunWith;\nimport org.junit.runners.JUnit4;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\n@RunWith(JUnit4.class)\npublic class Edition006_iOS_Upgrade {\n\n private String BUNDLE_ID = \"io.cloudgrey.the-app\";\n\n private String APP_V1_0_0 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.0/TheApp-v1.0.0.app.zip\";\n private String APP_V1_0_1 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.1/TheApp-v1.0.1.app.zip\";\n private String APP_V1_0_2 = \"https://github.com/cloudgrey-io/the-app/releases/download/v1.0.2/TheApp-v1.0.2.app.zip\";\n\n private By msgInput = By.xpath(\"//XCUIElementTypeTextField[@name=\\\"messageInput\\\"]\");\n private By savedMsg = MobileBy.AccessibilityId(\"savedMessage\");\n private By saveMsgBtn = MobileBy.AccessibilityId(\"messageSaveBtn\");\n private By echoBox = MobileBy.AccessibilityId(\"Echo Box\");\n\n private String TEST_MESSAGE = \"Hello World\";\n\n @Test\n public void testSavedTextAfterUpgrade () throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"deviceName\", \"iPhone 7\");\n capabilities.setCapability(\"platformVersion\", \"11.2\");\n capabilities.setCapability(\"app\", APP_V1_0_0);\n\n // change this to APP_V1_0_1 to experience a failing scenario\n String appUpgradeVersion = APP_V1_0_2;\n\n // Open the app.\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n wait.until(ExpectedConditions.presenceOfElementLocated(echoBox)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(msgInput)).sendKeys(TEST_MESSAGE);\n wait.until(ExpectedConditions.presenceOfElementLocated(saveMsgBtn)).click();\n String savedText = wait.until(ExpectedConditions.presenceOfElementLocated(savedMsg)).getText();\n Assert.assertEquals(savedText, TEST_MESSAGE);\n\n HashMap bundleArgs = new HashMap<>();\n bundleArgs.put(\"bundleId\", BUNDLE_ID);\n driver.executeScript(\"mobile: terminateApp\", bundleArgs);\n\n HashMap installArgs = new HashMap<>();\n installArgs.put(\"app\", appUpgradeVersion);\n driver.executeScript(\"mobile: installApp\", installArgs);\n\n driver.executeScript(\"mobile: launchApp\", bundleArgs);\n\n wait.until(ExpectedConditions.presenceOfElementLocated(echoBox)).click();\n savedText = wait.until(ExpectedConditions.presenceOfElementLocated(savedMsg)).getText();\n Assert.assertEquals(savedText, TEST_MESSAGE);\n } finally {\n driver.quit();\n }\n }\n}\n```\n\nYou can also find the code inside a working repository [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition006_iOS_Upgrade.java). That's it for testing app upgrades on iOS! Stay tuned for a future edition where we discuss how to achieve the same results with Android.\n","metadata":{"lang":"Java","platform":"iOS","subPlatform":"All Devices","title":"Testing iOS App Upgrades","shortDesc":"It's important to test that user data and user experiences are not broken due to changes between app version upgrades. Appium provides some helpful commands to test iOS app upgrades all within the context of one Appium session.","liveAt":"2018-02-21 06:00","canonicalRef":"https://www.headspin.io/blog/testing-ios-app-upgrades"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/6-testing-ios-app-upgrades","slug":"6-testing-ios-app-upgrades","path":"/editions/6-testing-ios-app-upgrades","news":null,"tags":["Java","iOS","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0005.md","num":5,"rawMd":"\n\nTraditionally, Appium is used to make functional verifications of mobile apps. What I mean is that people usually check the state of an app's UI to ensure that the app's features are working correctly. Functional testing is very important, but there are other dimensions of the user experience that are equally worth checking via automated means in your build. One of these other dimensions is performance. Performance is simply how responsive your app is to the user, and can include a variety of specific factors, from network request time to CPU an d memory usage.\n\nMobile apps, more than desktop apps, run in very resource-constrained environments. Mobile apps also have the potential not only to create a bad experience for the user while the app is open, but by hogging CPU or memory, could shorten the battery life or cause other applications to run slowly. Engaging in performance testing is therefore a way to be a good citizen of the app ecosystem, not just to ensure the snappiest experience for your own users.\n\nLuckily for Appium users, all kinds of interesting performance data is available to you via the Appium API---well, on Android at least. Building on the wealth of information that comes via `adb dumpsys`, Appium provides a succinct overview of all aspects of your app's performance via the [`getPerformanceData`](https://appium.io/docs/en/commands/device/performance-data/get-performance-data/) command. The client call is simple:\n\n```java\ndriver.getPerformanceData(\"\", \"\", );\n```\n\nHere, `` is the package of your AUT (or any other app you wish to profile). `` is what kind of performance data you want. There is another handy driver command ([`getSupportedPerformanceDataTypes`](https://appium.io/docs/en/commands/device/performance-data/performance-data-types/)) which tells you which types are valid. For the time being, they are: `cpuinfo`, `memoryinfo`, `batteryinfo`, and `networkinfo`. Finally, `` is an integer denoting the number of seconds Appium will poll for performance data if it is not immediately available.\n\nThere's too much to dig into in one edition, so let's focus on a simple example involving memory usage. One problem encountered by many apps at some point in their history is a memory leak. Despite living in a garbage-collected environment, Android apps can still cause memory to be locked up in an unusable state. It's therefore important to test that your app is not using increasing amounts of memory over time, without good reason.\n\nLet's construct a simple scenario to illustrate this, which is easily portable to your own Android apps. Basically, we want to open up a view in our app, take a snapshot of the memory usage, and then wait. After a while, we take another snapshot. Then we make an assertion that the memory usage outlined in the second snapshot is not significantly more than what we found in the first snapshot. This is a simple test that we don't have a memory leak.\n\nAssuming we're using the good old ApiDemos app, our call now looks like:\n\n```java\nList> data = driver.getPerformanceData(\"io.appium.android.apis\", \"memoryinfo\", 10);\n```\n\nWhat is returned is a set of two lists; one list is the keys and the other is the values. We can bundle up the call above along with some helper code that makes it easier to query the specific kind of memory info we're looking for (of course, in the real world we might make a class to hold the data):\n\n```java\nprivate HashMap getMemoryInfo(AndroidDriver driver) throws Exception {\n List> data = driver.getPerformanceData(\"io.appium.android.apis\", \"memoryinfo\", 10);\n HashMap readableData = new HashMap<>();\n for (int i = 0; i < data.get(0).size(); i++) {\n int val;\n if (data.get(1).get(i) == null) {\n val = 0;\n } else {\n val = Integer.parseInt((String) data.get(1).get(i));\n }\n readableData.put((String) data.get(0).get(i), val);\n }\n return readableData;\n}\n```\n\nEssentially, we now have a `HashMap` we can use to query the particular kinds of memory info we retrieved. In our case, we're going to look for the `totalPss` value:\n\n```java\nHashMap memoryInfo = getMemoryInfo(driver);\nint setSize = memoryInfo.get(\"totalPss\");\n```\n\nWhat is this `totalPss` business? PSS means 'Proportional Set Size'. According to the [Android `dumpsys` docs](https://developer.android.com/studio/command-line/dumpsys.html):\n\n> This is a measurement of your app’s RAM use that takes into account sharing pages across processes. Any RAM pages that are unique to your process directly contribute to its PSS value, while pages that are shared with other processes contribute to the PSS value only in proportion to the amount of sharing. For example, a page that is shared between two processes will contribute half of its size to the PSS of each process.\n\nIn other words, it's a pretty good measurement of the RAM impact of our app. There's a lot more to dig into here, but this measurement will do for our case. All that remains is for us to tie this into the Appium boilerplate and make a real assertion on our app. Here's the full code for the test case we outlined above (also available [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition005_Android_Memory.java)):\n\n```java\nimport io.appium.java_client.android.AndroidDriver;\nimport java.io.File;\nimport java.net.URL;\nimport java.util.HashMap;\nimport java.util.List;\nimport org.hamcrest.Matchers;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition005_Android_Memory {\n\n private static int MEMORY_USAGE_WAIT = 30000;\n private static int MEMORY_CAPTURE_WAIT = 10;\n private static String PKG = \"io.appium.android.apis\";\n private static String PERF_TYPE = \"memoryinfo\";\n private static String PSS_TYPE = \"totalPss\";\n\n @Test\n public void testMemoryUsage() throws Exception {\n File classpathRoot = new File(System.getProperty(\"user.dir\"));\n File appDir = new File(classpathRoot, \"../apps/\");\n File app = new File(appDir.getCanonicalPath(), \"ApiDemos.apk\");\n\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"app\", app);\n\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n try {\n // get the usage at one point in time\n int totalPss1 = getMemoryInfo(driver).get(PSS_TYPE);\n\n // then get it again after waiting a while\n try { Thread.sleep(MEMORY_USAGE_WAIT); } catch (InterruptedException ign) {}\n int totalPss2 = getMemoryInfo(driver).get(PSS_TYPE);\n\n // finally, verify that we haven't increased usage more than 5%\n Assert.assertThat((double) totalPss2, Matchers.lessThan(totalPss1 * 1.05));\n } finally {\n driver.quit();\n }\n }\n\n private HashMap getMemoryInfo(AndroidDriver driver) throws Exception {\n List> data = driver.getPerformanceData(PKG, PERF_TYPE, MEMORY_CAPTURE_WAIT);\n HashMap readableData = new HashMap<>();\n for (int i = 0; i < data.get(0).size(); i++) {\n int val;\n if (data.get(1).get(i) == null) {\n val = 0;\n } else {\n val = Integer.parseInt((String) data.get(1).get(i));\n }\n readableData.put((String) data.get(0).get(i), val);\n }\n return readableData;\n }\n}\n```\n\nI heartily recommend throwing some automated performance testing in the mix for your app. With even a basic test like the one above, you could catch issues that greatly affect your user's experiences, even if a functional test wouldn't have noticed anything wrong. Stay tuned for more discussion on performance testing or other kinds of testing made possible by Appium, in future editions.\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Performance Testing of Android Apps","shortDesc":"Functional testing is not the only kind of testing that Appium supports. As a pure automation library, Appium can also be used in the service of performance testing. In this edition we take a look at an example of performance testing on Android, namely how to track memory usage over time and make assertions about it.","liveAt":"2018-02-14 06:00","canonicalRef":"https://www.headspin.io/blog/performance-testing-of-android-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/5-performance-testing-of-android-apps","slug":"5-performance-testing-of-android-apps","path":"/editions/5-performance-testing-of-android-apps","news":{"rawMd":"* I will be delivering a webinar in a few weeks with some brand new slides on mobile testing, including one slide with a massive chart! (I won't tell you what's on the chart, but believe me it's massive). The webinar is being hosted by the good folks at [Applitools](https://applitools.com), and you need to [sign up](http://go.applitools.com/180223-Mobile-Automation-Webinar-w-J-Lipps.html) to attend.\n* There's still time to register for [SauceCon 2018](https://saucecon.com/), where I will be presenting on the various new Appium drivers. I've got something pretty fun planned for this one!\n","num":5,"mdPath":"/vercel/path0/content/news/0005.md"},"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0004.md","num":4,"rawMd":"\n\nMost people associate Appium with native mobile apps. That's how Appium started, and it's still Appium's core value proposition. However, Appium allows you to test a variety of apps---not just native. There are basically three kinds of mobile apps:\n\n1. *Native apps*. These are apps which are written and built using the OS-provided SDK and native APIs. Users get these apps from an official app store and run them by tapping on the app icon from the home screen of their mobile device.\n2. *Web apps*. These are apps which are written in HTML/JS/CSS and deployed via a web server. Users access them by navigating to a URL in the mobile browser of their choice (e.g., Safari, Chrome).\n3. *Hybrid apps*. These are apps which mix the previous two modes into the same app. Hybrid apps have a native \"shell\", which might involve a fair amount of native UI, or it might involve no native UI at all. Stuck into one or more of the app screens is a UI component called a \"web view\", which is essentially a chromeless embedded web browser. These web views can access local or remote URLs. Some hybrid apps build the entire set of app functionality in HTML/JS/CSS which are bundled with the native app package, and retrieved locally by the webview. Other hybrid apps access remote URLs. There are a lot of possibilities for hybrid app architecture. Regardless of how they're set up, hybrid apps have two modes---native and web. Appium can access both! (More on hybrid apps in a future edition).\n\nAppium enables you to automate any kind of app, across both iOS and Android platforms. The only difference is in how you set up the desired capabilities, and then in the commands you have access to once the session is started. This is where Appium's dependence on the WebDriver protocol really shines: an Appium-based mobile web test is just the same thing as a Selenium test! In fact, you can even use a standard Selenium client to speak to the Appium server and automate a mobile web app. The key is to use the `browserName` capability instead of the `app` capability. Appium will then take care of launching the specified browser and getting you automatically into the web context so you have access to all the regular Selenium methods you're used to (like finding elements by CSS, navigating to URLs, etc...).\n\nThe nice thing about testing mobile apps is that there is no difference between platforms in how your test is written. In the same way that you would expect the code for your Selenium tests to remain the same, regardless of whether you're testing Firefox or Chrome, your Appium web tests remain the same regardless of whether you're testing Safari on iOS or Chrome on Android. Let's take a look at what the capabilities would look like to start a session on these two mobile browsers:\n\n```java\n// test Safari on iOS\nDesiredCapabilities capabilities = new DesiredCapabilities();\ncapabilities.setCapability(\"platformName\", \"iOS\");\ncapabilities.setCapability(\"platformVersion\", \"11.2\");\ncapabilities.setCapability(\"deviceName\", \"iPhone 7\");\ncapabilities.setCapability(\"browserName\", \"Safari\");\n\n// test Chrome on Android\nDesiredCapabilities capabilities = new DesiredCapabilities();\ncapabilities.setCapability(\"platformName\", \"Android\");\ncapabilities.setCapability(\"deviceName\", \"Android Emulator\");\ncapabilities.setCapability(\"browserName\", \"Chrome\");\n```\n\nFrom here on out, after we've launched the appropriate `RemoteWebDriver` session (or `IOSDriver` or `AndroidDriver` if you have the Appium client), we can do whatever we want. For example, we could navigate to a website and get its title!\n\n```java\ndriver.get(\"https://appiumpro.com\");\nString title = driver.getTitle();\n// here we might verify that the title contains \"Appium Pro\"\n```\n\n### Special notes for iOS real devices\n\nThe only caveat to be aware of is with iOS real devices. Because of the way Appium talks to Safari (via the remote debugger exposed by the browser) an extra step is required to translate the WebKit remote debug protocol to Apple's iOS web inspector protocol exposed by usbmuxd. Sound complicated? Thankfully, the good folks at Google have created a tool that enables this translation, called [ios-webkit-debug-proxy](https://github.com/google/ios-webkit-debug-proxy) (IWDP). For running Safari tests on a real device, or hybrid tests on a real device, IWDP must be installed on your system. For more info on how to do that, you can check out the [Appium IWDP doc](https://appium.io/docs/en/writing-running-appium/web/ios-webkit-debug-proxy/). Once you've got IWDP installed, you simply need to add one two more capabilities to the set for iOS above, `udid` and `startIWDP`:\n\n```java\n// extra capabilities for Safari on a real iOS device\ncapabilities.setCapability(\"udid\", \"\");\ncapabilities.setCapability(\"startIWDP\", true);\n```\n\nIf you dont include the `startIWDP` capability, you must run IWDP on your own and Appium will just assume it's there listening for proxy requests.\n\n_Note that running tests on real iOS devices is a whole topic in and of itself, which I will cover in more detail in a future edition of Appium Pro. Meanwhile you can refer to the [real device documentation](https://appium.io/docs/en/drivers/ios-xcuitest-real-devices/) available at the Appium docs site._\n\n### Special notes for Android\n\nThankfully, things are simpler in the Android world because for both emulators and real devices Appium can take advantage of [Chromedriver](https://sites.google.com/a/chromium.org/chromedriver/). When you want to automate any Chrome-based browser or webview, Appium simply manages a new Chromedriver process under the hood, so you get the full power of a first-class WebDriver server without having to set anything up yourself. This does mean, however, that you may need to ensure that the version of Chrome on your Android system is compatible with the version of Chromedriver used by Appium. If it's not, you'll get a pretty obvious error message saying you need to upgrade Chrome.\n\nIf you don't want to upgrade Chrome, you can actually tell Appium which version of Chromedriver you want installed, when you install Appium, using the `--chromedriver_version` flag. For example:\n\n```\nnpm install -g appium --chromedriver_version=\"2.35\"\n```\n\nHow do you know which version of Chromedriver to install? The Appium team maintains a helpful list of which versions of Chromedriver support which versions of Chrome at our [Chromedriver guide](https://appium.io/docs/en/writing-running-appium/web/chromedriver/) in the docs.\n\n### A full example\n\nLet's take advantage of the fact that we can run web tests on both iOS and Android without any code changes, and construct a scenario for testing the Appium Pro contact submission form. We want to make sure that if a user tries to submit it without going through the Captcha challenge, an appropriate error message is presented. Here's the codefor the actual test logic that we care about:\n\n```java\ndriver.get(\"http://appiumpro.com/contact\");\nwait.until(ExpectedConditions.visibilityOfElementLocated(EMAIL))\n .sendKeys(\"foo@foo.com\");\ndriver.findElement(MESSAGE).sendKeys(\"Hello!\");\ndriver.findElement(SEND).click();\nString response = wait.until(ExpectedConditions.visibilityOfElementLocated(ERROR)).getText();\n\n// validate that we get an error message involving a captcha, which we didn't fill out\nAssert.assertThat(response, CoreMatchers.containsString(\"Captcha\"));\n```\n\nYou can see that I've put some selectors into class variables for readability. This is basically all we need to have a useful test! Though of course we do need some boilerplate to set up the desired capabilities for iOS and Android. The full test file looks like:\n\n```java\nimport io.appium.java_client.AppiumDriver;\nimport io.appium.java_client.android.AndroidDriver;\nimport io.appium.java_client.ios.IOSDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport org.hamcrest.CoreMatchers;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\npublic class Edition004_Web_Testing {\n\n private static By EMAIL = By.id(\"contactEmail\");\n private static By MESSAGE = By.id(\"contactText\");\n private static By SEND = By.cssSelector(\"input[type=submit]\");\n private static By ERROR = By.cssSelector(\".contactResponse\");\n\n @Test\n public void testAppiumProSite_iOS() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"iOS\");\n capabilities.setCapability(\"platformVersion\", \"11.2\");\n capabilities.setCapability(\"deviceName\", \"iPhone 7\");\n capabilities.setCapability(\"browserName\", \"Safari\");\n\n // Open up Safari\n IOSDriver driver = new IOSDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n\n @Test\n public void testAppiumProSite_Android() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"browserName\", \"Chrome\");\n\n // Open up Safari\n AndroidDriver driver = new AndroidDriver(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n actualTest(driver);\n }\n\n public void actualTest(AppiumDriver driver) {\n // Set up default wait\n WebDriverWait wait = new WebDriverWait(driver, 10);\n\n try {\n driver.get(\"http://appiumpro.com/contact\");\n wait.until(ExpectedConditions.visibilityOfElementLocated(EMAIL))\n .sendKeys(\"foo@foo.com\");\n driver.findElement(MESSAGE).sendKeys(\"Hello!\");\n driver.findElement(SEND).click();\n String response = wait.until(ExpectedConditions.visibilityOfElementLocated(ERROR)).getText();\n\n // validate that we get an error message involving a captcha, which we didn't fill out\n Assert.assertThat(response, CoreMatchers.containsString(\"Captcha\"));\n } finally {\n driver.quit();\n }\n\n }\n}\n```\n\nThis is all you need to get started with web testing in Appium. As always, you can find the [source for this edition](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition004_Web_Testing.java) in a working repo on GitHub.\n","metadata":{"lang":"Java","platform":"iOS and Android","subPlatform":"All Devices","title":"Using Appium for Testing Mobile Web Apps","shortDesc":"Appium is not just for native apps. One of the great things about Appium is being able to test multiple app modes. You can even use Appium to run tests against regular old web applications on mobile devices.","liveAt":"2018-02-07 06:00","canonicalRef":"https://www.headspin.io/blog/using-appium-for-testing-mobile-web-apps"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/4-using-appium-for-testing-mobile-web-apps","slug":"4-using-appium-for-testing-mobile-web-apps","path":"/editions/4-using-appium-for-testing-mobile-web-apps","news":{"rawMd":"This week was all about web testing. Did you know that there's a great newsletter called [Elemental Selenium](http://elementalselenium.com) that's all about using Selenium for web testing? Because Appium and Selenium are so similar, many of that newsletter's tips are also really useful food for thought if you're doing web or hybrid testing using Appium. Check it out!\n","num":4,"mdPath":"/vercel/path0/content/news/0004.md"},"tags":["Java","iOS and Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0003.md","num":3,"rawMd":"\n\nPreviously, we discussed [how to get photos into the Android media library](https://appiumpro.com/editions/2). That was all well and good, but there was a bit of clunkiness in how we set up the conditions for the test. We actually automated the UI in order to remove all the existing pictures at the beginning of the test. Wouldn't it be great if there were a way to do this instantly, as part of test setup? Luckily, there is!\n\nIf you're not a big Android person, you might not know about [ADB](https://developer.android.com/studio/command-line/adb.html), the \"Android Debug Bridge\". ADB is a powerful tool provided as part of the Android SDK by Google, that allows running all sorts of interesting commands on a connected emulator or device. One of these commands is `adb shell`, which gives you shell access to the device filesystem (including root access on emulators or rooted devices). `adb shell` is the perfect tool for solving the problem above, because with it we can run the `rm` command to remove any existing images from the SD card prior to our test.\n\nFor a long time, Appium did not allow running of arbitrary ADB commands. This is because Appium was designed to run in a remote environment, possibly sharing an OS with other services or Appium servers, and potentially many connected Android devices. It would be a huge security hole to give any Appium client the full power of ADB in this context. Recently, the Appium team decided to unlock this functionality behind a special server flag, so that someone running an Appium server could intentionally open up this security hole (in situations where they know they're the only one using the server, for example).\n\nAs of Appium 1.7.2, the `--relaxed-security` flag is available, so you can now start up Appium like this:\n\n```sh\nappium --relaxed-security\n```\n\nWith Appium running in this mode, you have access to a new \"mobile:\" command called \"mobile: shell\". The Appium \"mobile:\" commands are special commands that can be accessed using `executeScript` (at least until client libraries make a nicer interface for taking advantage of them). Here's how a call to \"mobile: shell\" looks in Java:\n\n```java\ndriver.executeScript(\"mobile: shell\", );\n```\n\nWhat is `arg` here? It needs to be a JSONifiable object with two keys:\n\n1. `command`: a String, the command to be run under `adb shell`\n2. `args`: an array of Strings, the arguments passed to the shell command.\n\nFor the purposes of our example, let's say we want to clear out the pictures on the SD card, and that on our device, these are located at `/mnt/sdcard/Pictures`. If we were running ADB on our own without Appium, we'd accomplish our goal by running:\n\n```sh\nadb shell rm -rf /mnt/sdcard/Pictures/*.*\n```\n\nTo translate this to Appium's \"mobile: shell\" command, we simply strip off `adb shell` from the beginning, and we are left with:\n\n```sh\nrm -rf /mnt/sdcard/Pictures/*.*\n```\n\nThe first word here is the \"command\", and the rest constitute the \"args\". So we can construct our object as follows:\n\n```java\nList removePicsArgs = Arrays.asList(\n \"-rf\",\n \"/mnt/sdcard/Pictures/*.*\"\n);\nMap removePicsCmd = ImmutableMap.of(\n \"command\", \"rm\",\n \"args\", removePicsArgs\n);\ndriver.executeScript(\"mobile: shell\", removePicsCmd);\n```\n\nEssentially, we construct an Object that in JSON would look like:\n\n```json\n{\"command\": \"rm\", \"args\": [\"-rf\", \"/mnt/sdcard/Pictures/*.*\"]}\n```\n\nWe can also retrieve the result of the ADB call, for example if we wish to verify that the directory is now indeed empty:\n\n```java\nList lsArgs = Arrays.asList(\"/mnt/sdcard/Pictures/*.*\");\nMap lsCmd = ImmutableMap.of(\n \"command\", \"ls\",\n \"args\", lsArgs\n);\nString lsOutput = (String) driver.executeScript(\"mobile: shell\", lsCmd);\n```\n\nThe output of our command is returned to us as a String which we can do whatever we want with, including making assertions on it. Putting it all together now, here's a full test that uses Appium to combine the two actions above. Of course, it would be most useful in the context of some other actions, like the ones in the previous edition.\n\n```java\nimport com.google.common.collect.ImmutableMap;\nimport io.appium.java_client.android.AndroidDriver;\nimport java.net.MalformedURLException;\nimport java.net.URL;\nimport java.util.Arrays;\nimport java.util.List;\nimport java.util.Map;\nimport org.junit.Assert;\nimport org.junit.Test;\nimport org.openqa.selenium.remote.DesiredCapabilities;\n\npublic class Edition003_Arbitrary_ADB {\n\n private static String ANDROID_PHOTO_PATH = \"/mnt/sdcard/Pictures\";\n\n @Test\n public void testArbitraryADBCommands() throws MalformedURLException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"appPackage\", \"com.google.android.apps.photos\");\n capabilities.setCapability(\"appActivity\", \".home.HomeActivity\");\n\n // Open the app.\n AndroidDriver driver = new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n try {\n List removePicsArgs = Arrays.asList(\"-rf\", ANDROID_PHOTO_PATH + \"/*.*\");\n Map removePicsCmd = ImmutableMap\n .of(\"command\", \"rm\", \"args\", removePicsArgs);\n driver.executeScript(\"mobile: shell\", removePicsCmd);\n\n List lsArgs = Arrays.asList(\"/mnt/sdcard\");\n Map lsCmd = ImmutableMap.of(\"command\", \"ls\", \"args\", lsArgs);\n String lsOutput = (String) driver.executeScript(\"mobile: shell\", lsCmd);\n Assert.assertEquals(\"\", lsOutput);\n } finally {\n driver.quit();\n }\n }\n\n}\n```\n\nIn this edition we've explored the power of ADB through a few simple filesystem commands. You can actually do many more useful things than delete files with ADB, so go out there and have fun with it. Let me know if you come up with any ingenious uses! As always, you can check out the code in the context of all its dependencies [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition003_Arbitrary_ADB.java).\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Running arbitrary ADB commands via Appium","shortDesc":"For a long time, it was not possible to run arbitrary ADB commands with Appium. Now it is! Here's how to remove the bumpers and get dangerous with ADB.","liveAt":"2018-01-31 08:00","newsletterTopNote":"This week I'm bringing back an old but useful post, since I'm on vacation! Enjoy.","canonicalRef":"https://www.headspin.io/blog/running-arbitrary-adb-commands-via-appium"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/3-running-arbitrary-adb-commands-via-appium","slug":"3-running-arbitrary-adb-commands-via-appium","path":"/editions/3-running-arbitrary-adb-commands-via-appium","news":{"rawMd":"","num":3,"mdPath":"/vercel/path0/content/news/0003.md"},"tags":["Java","Android","All Devices"]},{"mdPath":"/vercel/path0/content/editions/0002.md","num":2,"rawMd":"\n\nAs promised, this week's edition is an Android-flavored follow-up to last week's tip on [seeding the iOS simulator with test photos](https://appiumpro.com/editions/1). The problem we're trying to solve is how to get pictures with known content onto the device for use in our App Under Test. This is a requirement for any type of app that utilizes or processes images.\n\nHow do you test that your app does the right thing to a user-provided image?\n\n1. Take some image you have lying around, and input it manually to your application.\n2. Manually run the app function on your image. Maybe this is applying a certain type of filter, for example.\n3. Still manually, extract the modified image from your app any way you can (texting it to yourself, for example!)\n\nWhat these steps do is provide a gold-standard before-and-after which you can use as a test fixture. In an automated fashion now, we can provide the app with the same initial picture, run the desired function, and then retrieve the modified image. We can verify this modified image is byte-for-byte equivalent to our gold standard to ensure the app functionality still works as expected.\n\nIn this edition we focus on the problem of getting our initial picture onto the device. How does one do this for Android? Happily, we use the same function as for iOS: `pushFile`. Under the hood, `pushFile` uses a series of `ADB` commands to shuffle the image to the device and then broadcast a system intent to refresh the media library. Since different device manufacturers put pictures in different places, you do need to know the path on your device that stores media. For emulators and at least some real devices, the path on the device is `/mnt/sdcard/Pictures`, so this is an important constant to your remember. As an example:\n\n```java\ndriver.pushFile(\"/mnt/sdcard/Pictures/myPhoto.jpg\", \"/path/to/photo/locally/myPhoto.jpg\");\n```\n\nAs you can see, the first argument is the remote path on the device where we want the picture to end up. This is where you may need to refer to documentation on your particular device or Android OS flavor to ensure you have the right path. The second argument is the path to the file on your local machine, where your test is running. When the test executes, the Appium client will encode this file as a string and send it over to the Appium server, which will then do the job of getting it on the device. This architecture is nice because it means that `pushFile` works whether you're running locally, or on a cloud provider like [Sauce Labs](https://saucelabs.com).\n\nLet's take a look at a complete working example. In this example, we automate the built-in Google Photos app. First of all, we set up our desired capabilities to just use a built-in app:\n\n```java\nDesiredCapabilities capabilities = new DesiredCapabilities();\ncapabilities.setCapability(\"platformName\", \"Android\");\ncapabilities.setCapability(\"deviceName\", \"Android Emulator\");\ncapabilities.setCapability(\"automationName\", \"UiAutomator2\");\ncapabilities.setCapability(\"appPackage\", \"com.google.android.apps.photos\");\ncapabilities.setCapability(\"appActivity\", \".home.HomeActivity\");\n\n// Open the app.\nAndroidDriver driver = new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n```\n\nOnce the app is open, we have to navigate through some UI boilerplate and other things Google wants us to do to use their cloud. Of course we are robots and not interested in their cloud! I've put this app state setup into its own method, along with setting up some locator constants, and logic that removes any existing pictures to make our verification easier later on:\n\n```java\nprivate static By backupSwitch = By.id(\"com.google.android.apps.photos:id/auto_backup_switch\");\nprivate static By touchOutside = By.id(\"com.google.android.apps.photos:id/touch_outside\");\nprivate static By keepOff = By.xpath(\"//*[@text='KEEP OFF']\");\nprivate static By photo = By.xpath(\"//android.view.ViewGroup[contains(@content-desc, 'Photo taken')]\");\nprivate static By trash = By.id(\"com.google.android.apps.photos:id/trash\");\nprivate static By moveToTrash = By.xpath(\"//*[@text='MOVE TO TRASH']\");\n\npublic void setupAppState(AndroidDriver driver) {\n // navigate through the google junk to get to the app\n WebDriverWait wait = new WebDriverWait(driver, 10);\n WebDriverWait shortWait = new WebDriverWait(driver, 3);\n wait.until(ExpectedConditions.presenceOfElementLocated(backupSwitch)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(touchOutside)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(keepOff)).click();\n\n // delete any existing pictures using an infinite loop broken when we can't find any\n // more pictures\n try {\n while (true) {\n shortWait.until(ExpectedConditions.presenceOfElementLocated(photo)).click();\n shortWait.until(ExpectedConditions.presenceOfElementLocated(trash)).click();\n shortWait.until(ExpectedConditions.presenceOfElementLocated(moveToTrash)).click();\n }\n } catch (TimeoutException ignore) {}\n}\n```\n\nFinally we can get to the meat of our test, which relies upon the appropriate device media path constant:\n\n```java\nsetupAppState(driver);\n\n// set up the file we want to push to the phone's library\nFile assetDir = new File(classpathRoot, \"../assets\");\nFile img = new File(assetDir.getCanonicalPath(), \"cloudgrey.png\");\n\n// actually push the file\ndriver.pushFile(ANDROID_PHOTO_PATH + \"/\" + img.getName(), img);\n\n// wait for the system to acknowledge the new photo, and use the WebDriverWait to verify\n// that the new photo is there\nWebDriverWait wait = new WebDriverWait(driver, 10);\nExpectedCondition condition = ExpectedConditions.numberOfElementsToBe(photo,1);\nwait.until(condition);\n```\n\nAs you can see, the actual bit we care about is as simple as calling `pushFile`. For the sake of this test, we are simply verifying that our picture exists at the end. In a real-world scenario, we'd then run our app functionality on the picture, and retrieve it from the device to perform some local verification on it.\n\nBringing it all together, the entire test class looks like the following:\n\n```java\nimport io.appium.java_client.android.AndroidDriver;\nimport java.io.File;\nimport java.io.IOException;\nimport java.net.URL;\nimport org.junit.Test;\nimport org.openqa.selenium.By;\nimport org.openqa.selenium.TimeoutException;\nimport org.openqa.selenium.remote.DesiredCapabilities;\nimport org.openqa.selenium.support.ui.ExpectedCondition;\nimport org.openqa.selenium.support.ui.ExpectedConditions;\nimport org.openqa.selenium.support.ui.WebDriverWait;\n\n// Note that to function correctly, this test must be run against a version of Appium which includes\n// the appium-android-driver package at version 1.38 or higher, since it contains relevant bugfixes\n\npublic class Edition002_Android_Photos {\n\n private static String ANDROID_PHOTO_PATH = \"/mnt/sdcard/Pictures\";\n\n private static By backupSwitch = By.id(\"com.google.android.apps.photos:id/auto_backup_switch\");\n private static By touchOutside = By.id(\"com.google.android.apps.photos:id/touch_outside\");\n private static By keepOff = By.xpath(\"//*[@text='KEEP OFF']\");\n private static By photo = By.xpath(\"//android.view.ViewGroup[contains(@content-desc, 'Photo taken')]\");\n private static By trash = By.id(\"com.google.android.apps.photos:id/trash\");\n private static By moveToTrash = By.xpath(\"//*[@text='MOVE TO TRASH']\");\n\n @Test\n public void testSeedPhotoPicker() throws IOException {\n DesiredCapabilities capabilities = new DesiredCapabilities();\n\n File classpathRoot = new File(System.getProperty(\"user.dir\"));\n\n capabilities.setCapability(\"platformName\", \"Android\");\n capabilities.setCapability(\"deviceName\", \"Android Emulator\");\n capabilities.setCapability(\"automationName\", \"UiAutomator2\");\n capabilities.setCapability(\"appPackage\", \"com.google.android.apps.photos\");\n capabilities.setCapability(\"appActivity\", \".home.HomeActivity\");\n\n // Open the app.\n AndroidDriver driver = new AndroidDriver<>(new URL(\"http://localhost:4723/wd/hub\"), capabilities);\n\n try {\n // there's some screens we need to navigate through and ensure there are no existing photos\n setupAppState(driver);\n\n // set up the file we want to push to the phone's library\n File assetDir = new File(classpathRoot, \"../assets\");\n File img = new File(assetDir.getCanonicalPath(), \"cloudgrey.png\");\n\n // actually push the file\n driver.pushFile(ANDROID_PHOTO_PATH + \"/\" + img.getName(), img);\n\n // wait for the system to acknowledge the new photo, and use the WebDriverWait to verify\n // that the new photo is there\n WebDriverWait wait = new WebDriverWait(driver, 10);\n ExpectedCondition condition = ExpectedConditions.numberOfElementsToBe(photo,1);\n wait.until(condition);\n } finally {\n driver.quit();\n }\n }\n\n public void setupAppState(AndroidDriver driver) {\n // navigate through the google junk to get to the app\n WebDriverWait wait = new WebDriverWait(driver, 10);\n WebDriverWait shortWait = new WebDriverWait(driver, 3);\n wait.until(ExpectedConditions.presenceOfElementLocated(backupSwitch)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(touchOutside)).click();\n wait.until(ExpectedConditions.presenceOfElementLocated(keepOff)).click();\n\n // delete any existing pictures using an infinite loop broken when we can't find any\n // more pictures\n try {\n while (true) {\n shortWait.until(ExpectedConditions.presenceOfElementLocated(photo)).click();\n shortWait.until(ExpectedConditions.presenceOfElementLocated(trash)).click();\n shortWait.until(ExpectedConditions.presenceOfElementLocated(moveToTrash)).click();\n }\n } catch (TimeoutException ignore) {}\n }\n}\n```\n\nAs always, you can check out the code in the context of all its dependencies [on GitHub](https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition002_Android_Photos.java)!\n","metadata":{"lang":"Java","platform":"Android","subPlatform":"All Devices","title":"Seeding an Android device with test photos","shortDesc":"In this edition we explore the Android equivalent to our previous tip: how to add your own photos to the device so they can be available to media apps.","liveAt":"2018-01-24 08:00","canonicalRef":"https://www.headspin.io/blog/seeding-an-android-device-with-test-photos"},"mtime":"2023-04-28T16:49:48Z","permalink":"https://appiumpro.com/editions/2-seeding-an-android-device-with-test-photos","slug":"2-seeding-an-android-device-with-test-photos","path":"/editions/2-seeding-an-android-device-with-test-photos","news":{"rawMd":"* Did you know that Appium is having its very own conference? The first-ever AppiumConf will be held on April 6 in London, with a selection of workshops running the day before. A ton of awesome speakers have been announced, including Jonathan Lipps, Dan Cuellar, Kristel Kruustuk, and more. Check it out and buy tickets at [appiumconf.com](https://appiumconf.com)!\n* A one-day Appium meetup and workshop is being held on February 10 in Singapore. It's being organized by Carousell. If you're in that part of the world, [check it out](https://www.eventbrite.sg/e/singapore-appium-meetup-1-day-workshop-tickets-41695668868).\n","num":2,"mdPath":"/vercel/path0/content/news/0002.md"},"tags":["Java","Android","All Devices"]}],"tag":"All Devices"},"__N_SSG":true}