| Scott Main | 50e990c | 2012-06-21 17:14:39 -0700 | [diff] [blame] | 1 | page.title=Testing Fundamentals |
| 2 | parent.title=Testing |
| 3 | parent.link=index.html |
| 4 | @jd:body |
| 5 | |
| 6 | <div id="qv-wrapper"> |
| 7 | <div id="qv"> |
| 8 | <h2>In this document</h2> |
| 9 | <ol> |
| 10 | <li> |
| 11 | <a href="#TestStructure">Test Structure</a> |
| 12 | </li> |
| 13 | <li> |
| 14 | <a href="#TestProjects">Test Projects</a> |
| 15 | </li> |
| 16 | <li> |
| 17 | <a href="#TestAPI">The Testing API</a> |
| 18 | <ol> |
| 19 | <li> |
| 20 | <a href="#JUnit">JUnit</a> |
| 21 | </li> |
| 22 | <li> |
| 23 | <a href="#Instrumentation">Instrumentation</a> |
| 24 | </li> |
| 25 | <li> |
| 26 | <a href="#TestCaseClasses">Test case classes</a> |
| 27 | </li> |
| 28 | <li> |
| 29 | <a href="#AssertionClasses">Assertion classes</a> |
| 30 | </li> |
| 31 | <li> |
| 32 | <a href="#MockObjectClasses">Mock object classes</a> |
| 33 | </li> |
| 34 | </ol> |
| 35 | </li> |
| 36 | <li> |
| 37 | <a href="#InstrumentationTestRunner">Running Tests</a> |
| 38 | </li> |
| 39 | <li> |
| 40 | <a href="#TestResults">Seeing Test Results</a> |
| 41 | </li> |
| 42 | <li> |
| 43 | <a href="#Monkeys">monkey and monkeyrunner</a> |
| 44 | </li> |
| 45 | <li> |
| 46 | <a href="#PackageNames">Working With Package Names</a> |
| 47 | </li> |
| 48 | <li> |
| 49 | <a href="#WhatToTest">What To Test</a> |
| 50 | </li> |
| 51 | <li> |
| 52 | <a href="#NextSteps">Next Steps</a> |
| 53 | </li> |
| 54 | </ol> |
| 55 | <h2>Key classes</h2> |
| 56 | <ol> |
| 57 | <li>{@link android.test.InstrumentationTestRunner}</li> |
| 58 | <li>{@link android.test}</li> |
| 59 | <li>{@link android.test.mock}</li> |
| 60 | <li>{@link junit.framework}</li> |
| 61 | </ol> |
| 62 | <h2>Related tutorials</h2> |
| 63 | <ol> |
| 64 | <li> |
| 65 | <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a> |
| 66 | </li> |
| 67 | </ol> |
| 68 | <h2>See also</h2> |
| 69 | <ol> |
| 70 | <li> |
| 71 | <a href="{@docRoot}tools/testing/testing_eclipse.html"> |
| 72 | Testing from Eclipse with ADT</a> |
| 73 | </li> |
| 74 | <li> |
| 75 | <a href="{@docRoot}tools/testing/testing_otheride.html"> |
| 76 | Testing from Other IDEs</a> |
| 77 | </li> |
| 78 | <li> |
| 79 | <a href="{@docRoot}tools/help/monkeyrunner_concepts.html"> |
| 80 | monkeyrunner</a> |
| 81 | </li> |
| 82 | <li> |
| 83 | <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a> |
| 84 | </li> |
| 85 | </ol> |
| 86 | </div> |
| 87 | </div> |
| 88 | <p> |
| 89 | The Android testing framework, an integral part of the development environment, |
| 90 | provides an architecture and powerful tools that help you test every aspect of your application |
| 91 | at every level from unit to framework. |
| 92 | </p> |
| 93 | <p> |
| 94 | The testing framework has these key features: |
| 95 | </p> |
| 96 | <ul> |
| 97 | <li> |
| 98 | Android test suites are based on JUnit. You can use plain JUnit to test a class that doesn't |
| 99 | call the Android API, or Android's JUnit extensions to test Android components. If you're |
| 100 | new to Android testing, you can start with general-purpose test case classes such as {@link |
| 101 | android.test.AndroidTestCase} and then go on to use more sophisticated classes. |
| 102 | </li> |
| 103 | <li> |
| 104 | The Android JUnit extensions provide component-specific test case classes. These classes |
| 105 | provide helper methods for creating mock objects and methods that help you control the |
| 106 | lifecycle of a component. |
| 107 | </li> |
| 108 | <li> |
| 109 | Test suites are contained in test packages that are similar to main application packages, so |
| 110 | you don't need to learn a new set of tools or techniques for designing and building tests. |
| 111 | </li> |
| 112 | <li> |
| 113 | The SDK tools for building and tests are available in Eclipse with ADT, and also in |
| kmccormick | 3b9f0aa | 2013-04-01 18:08:28 -0700 | [diff] [blame] | 114 | command-line form for use with other IDEs. These tools get information from the project of |
| Scott Main | 50e990c | 2012-06-21 17:14:39 -0700 | [diff] [blame] | 115 | the application under test and use this information to automatically create the build files, |
| 116 | manifest file, and directory structure for the test package. |
| 117 | </li> |
| 118 | <li> |
| 119 | The SDK also provides |
| 120 | <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a>, an API |
| kmccormick | 3b9f0aa | 2013-04-01 18:08:28 -0700 | [diff] [blame] | 121 | for testing devices with Python programs, and <a |
| Scott Main | 50e990c | 2012-06-21 17:14:39 -0700 | [diff] [blame] | 122 | href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>, |
| 123 | a command-line tool for stress-testing UIs by sending pseudo-random events to a device. |
| 124 | </li> |
| 125 | </ul> |
| 126 | <p> |
| 127 | This document describes the fundamentals of the Android testing framework, including the |
| 128 | structure of tests, the APIs that you use to develop tests, and the tools that you use to run |
| 129 | tests and view results. The document assumes you have a basic knowledge of Android application |
| 130 | programming and JUnit testing methodology. |
| 131 | </p> |
| 132 | <p> |
| 133 | The following diagram summarizes the testing framework: |
| 134 | </p> |
| 135 | <div style="width: 70%; margin-left:auto; margin-right:auto;"> |
| 136 | <a href="{@docRoot}images/testing/test_framework.png"> |
| 137 | <img src="{@docRoot}images/testing/test_framework.png" |
| 138 | alt="The Android testing framework"/> |
| 139 | </a> |
| 140 | </div> |
| 141 | <h2 id="TestStructure">Test Structure</h2> |
| 142 | <p> |
| 143 | Android's build and test tools assume that test projects are organized into a standard |
| 144 | structure of tests, test case classes, test packages, and test projects. |
| 145 | </p> |
| 146 | <p> |
| 147 | Android testing is based on JUnit. In general, a JUnit test is a method whose |
| 148 | statements test a part of the application under test. You organize test methods into classes |
| 149 | called test cases (or test suites). Each test is an isolated test of an individual module in |
| 150 | the application under test. Each class is a container for related test methods, although it |
| 151 | often provides helper methods as well. |
| 152 | </p> |
| 153 | <p> |
| 154 | In JUnit, you build one or more test source files into a class file. Similarly, in Android you |
| 155 | use the SDK's build tools to build one or more test source files into class files in an |
| 156 | Android test package. In JUnit, you use a test runner to execute test classes. In Android, you |
| 157 | use test tools to load the test package and the application under test, and the tools then |
| 158 | execute an Android-specific test runner. |
| 159 | </p> |
| 160 | <h2 id="TestProjects">Test Projects</h2> |
| 161 | <p> |
| 162 | Tests, like Android applications, are organized into projects. |
| 163 | </p> |
| 164 | <p> |
| 165 | A test project is a directory or Eclipse project in which you create the source code, manifest |
| 166 | file, and other files for a test package. The Android SDK contains tools for Eclipse with ADT |
| 167 | and for the command line that create and update test projects for you. The tools create the |
| 168 | directories you use for source code and resources and the manifest file for the test package. |
| 169 | The command-line tools also create the Ant build files you need. |
| 170 | </p> |
| 171 | <p> |
| 172 | You should always use Android tools to create a test project. Among other benefits, |
| 173 | the tools: |
| 174 | </p> |
| 175 | <ul> |
| 176 | <li> |
| 177 | Automatically set up your test package to use |
| 178 | {@link android.test.InstrumentationTestRunner} as the test case runner. You must use |
| 179 | <code>InstrumentationTestRunner</code> (or a subclass) to run JUnit tests. |
| 180 | </li> |
| 181 | <li> |
| 182 | Create an appropriate name for the test package. If the application |
| 183 | under test has a package name of <code>com.mydomain.myapp</code>, then the |
| 184 | Android tools set the test package name to <code>com.mydomain.myapp.test</code>. This |
| 185 | helps you identify their relationship, while preventing conflicts within the system. |
| 186 | </li> |
| 187 | <li> |
| 188 | Automatically create the proper build files, manifest file, and directory |
| 189 | structure for the test project. This helps you to build the test package without |
| 190 | having to modify build files and sets up the linkage between your test package and |
| 191 | the application under test. |
| 192 | The |
| 193 | </li> |
| 194 | </ul> |
| 195 | <p> |
| 196 | You can create a test project anywhere in your file system, but the best approach is to |
| 197 | add the test project so that its root directory <code>tests/</code> is at the same level |
| 198 | as the <code>src/</code> directory of the main application's project. This helps you find the |
| 199 | tests associated with an application. For example, if your application project's root directory |
| 200 | is <code>MyProject</code>, then you should use the following directory structure: |
| 201 | </p> |
| 202 | <pre class="classic no-pretty-print"> |
| 203 | MyProject/ |
| 204 | AndroidManifest.xml |
| 205 | res/ |
| 206 | ... (resources for main application) |
| 207 | src/ |
| 208 | ... (source code for main application) ... |
| 209 | tests/ |
| 210 | AndroidManifest.xml |
| 211 | res/ |
| 212 | ... (resources for tests) |
| 213 | src/ |
| 214 | ... (source code for tests) |
| 215 | </pre> |
| 216 | <h2 id="TestAPI">The Testing API</h2> |
| 217 | <p> |
| 218 | The Android testing API is based on the JUnit API and extended with a instrumentation |
| 219 | framework and Android-specific testing classes. |
| 220 | </p> |
| 221 | <h3 id="JUnit">JUnit</h3> |
| 222 | <p> |
| 223 | You can use the JUnit {@link junit.framework.TestCase TestCase} class to do unit testing on |
| 224 | a class that doesn't call Android APIs. <code>TestCase</code> is also the base class for |
| 225 | {@link android.test.AndroidTestCase}, which you can use to test Android-dependent objects. |
| 226 | Besides providing the JUnit framework, AndroidTestCase offers Android-specific setup, |
| 227 | teardown, and helper methods. |
| 228 | </p> |
| 229 | <p> |
| 230 | You use the JUnit {@link junit.framework.Assert} class to display test results. |
| 231 | The assert methods compare values you expect from a test to the actual results and |
| 232 | throw an exception if the comparison fails. Android also provides a class of assertions that |
| 233 | extend the possible types of comparisons, and another class of assertions for testing the UI. |
| 234 | These are described in more detail in the section <a href="#AssertionClasses"> |
| 235 | Assertion classes</a> |
| 236 | </p> |
| 237 | <p> |
| 238 | To learn more about JUnit, you can read the documentation on the |
| 239 | <a href="http://www.junit.org">junit.org</a> home page. |
| 240 | Note that the Android testing API supports JUnit 3 code style, but not JUnit 4. Also, you must |
| 241 | use Android's instrumented test runner {@link android.test.InstrumentationTestRunner} to run |
| 242 | your test case classes. This test runner is described in the |
| 243 | section <a href="#InstrumentationTestRunner">Running Tests</a>. |
| 244 | </p> |
| 245 | <h3 id="Instrumentation">Instrumentation</h3> |
| 246 | <p> |
| 247 | Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks |
| 248 | control an Android component independently of its normal lifecycle. They also control how |
| 249 | Android loads applications. |
| 250 | </p> |
| 251 | <p> |
| 252 | Normally, an Android component runs in a lifecycle determined by the system. For example, an |
| 253 | Activity object's lifecycle starts when the Activity is activated by an Intent. The object's |
| 254 | <code>onCreate()</code> method is called, followed by <code>onResume()</code>. When the user |
| 255 | starts another application, the <code>onPause()</code> method is called. If the Activity |
| 256 | code calls the <code>finish()</code> method, the <code>onDestroy()</code> method is called. |
| 257 | The Android framework API does not provide a way for your code to invoke these callback |
| 258 | methods directly, but you can do so using instrumentation. |
| 259 | </p> |
| 260 | <p> |
| 261 | Also, the system runs all the components of an application into the same |
| 262 | process. You can allow some components, such as content providers, to run in a separate process, |
| 263 | but you can't force an application to run in the same process as another application that is |
| 264 | already running. |
| 265 | </p> |
| 266 | <p> |
| 267 | With Android instrumentation, though, you can invoke callback methods in your test code. |
| 268 | This allows you to run through the lifecycle of a component step by step, as if you were |
| 269 | debugging the component. The following test code snippet demonstrates how to use this to |
| 270 | test that an Activity saves and restores its state: |
| 271 | </p> |
| 272 | <a name="ActivitySnippet"></a> |
| 273 | <pre> |
| 274 | // Start the main activity of the application under test |
| 275 | mActivity = getActivity(); |
| 276 | |
| 277 | // Get a handle to the Activity object's main UI widget, a Spinner |
| 278 | mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner01); |
| 279 | |
| 280 | // Set the Spinner to a known position |
| 281 | mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION); |
| 282 | |
| 283 | // Stop the activity - The onDestroy() method should save the state of the Spinner |
| 284 | mActivity.finish(); |
| 285 | |
| 286 | // Re-start the Activity - the onResume() method should restore the state of the Spinner |
| 287 | mActivity = getActivity(); |
| 288 | |
| 289 | // Get the Spinner's current position |
| 290 | int currentPosition = mActivity.getSpinnerPosition(); |
| 291 | |
| 292 | // Assert that the current position is the same as the starting position |
| 293 | assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition); |
| 294 | </pre> |
| 295 | <p> |
| 296 | The key method used here is |
| 297 | {@link android.test.ActivityInstrumentationTestCase2#getActivity()}, which is a |
| 298 | part of the instrumentation API. The Activity under test is not started until you call this |
| 299 | method. You can set up the test fixture in advance, and then call this method to start the |
| 300 | Activity. |
| 301 | </p> |
| 302 | <p> |
| 303 | Also, instrumentation can load both a test package and the application under test into the |
| 304 | same process. Since the application components and their tests are in the same process, the |
| 305 | tests can invoke methods in the components, and modify and examine fields in the components. |
| 306 | </p> |
| 307 | <h3 id="TestCaseClasses">Test case classes</h3> |
| 308 | <p> |
| 309 | Android provides several test case classes that extend {@link junit.framework.TestCase} and |
| 310 | {@link junit.framework.Assert} with Android-specific setup, teardown, and helper methods. |
| 311 | </p> |
| 312 | <h4 id="AndroidTestCase">AndroidTestCase</h4> |
| 313 | <p> |
| 314 | A useful general test case class, especially if you are |
| 315 | just starting out with Android testing, is {@link android.test.AndroidTestCase}. It extends |
| 316 | both {@link junit.framework.TestCase} and {@link junit.framework.Assert}. It provides the |
| 317 | JUnit-standard <code>setUp()</code> and <code>tearDown()</code> methods, as well as |
| 318 | all of JUnit's Assert methods. In addition, it provides methods for testing permissions, and a |
| 319 | method that guards against memory leaks by clearing out certain class references. |
| 320 | </p> |
| 321 | <h4 id="ComponentTestCase">Component-specific test cases</h4> |
| 322 | <p> |
| 323 | A key feature of the Android testing framework is its component-specific test case classes. |
| 324 | These address specific component testing needs with methods for fixture setup and |
| 325 | teardown and component lifecycle control. They also provide methods for setting up mock objects. |
| 326 | These classes are described in the component-specific testing topics: |
| 327 | </p> |
| 328 | <ul> |
| 329 | <li> |
| 330 | <a href="{@docRoot}tools/testing/activity_testing.html">Activity Testing</a> |
| 331 | </li> |
| 332 | <li> |
| 333 | <a href="{@docRoot}tools/testing/contentprovider_testing.html"> |
| 334 | Content Provider Testing</a> |
| 335 | </li> |
| 336 | <li> |
| 337 | <a href="{@docRoot}tools/testing/service_testing.html">Service Testing</a> |
| 338 | </li> |
| 339 | </ul> |
| 340 | <p> |
| 341 | Android does not provide a separate test case class for BroadcastReceiver. Instead, test a |
| 342 | BroadcastReceiver by testing the component that sends it Intent objects, to verify that the |
| 343 | BroadcastReceiver responds correctly. |
| 344 | </p> |
| 345 | <h4 id="ApplicationTestCase">ApplicationTestCase</h4> |
| 346 | <p> |
| 347 | You use the {@link android.test.ApplicationTestCase} test case class to test the setup and |
| 348 | teardown of {@link android.app.Application} objects. These objects maintain the global state of |
| 349 | information that applies to all the components in an application package. The test case can |
| 350 | be useful in verifying that the <application> element in the manifest file is correctly |
| 351 | set up. Note, however, that this test case does not allow you to control testing of the |
| 352 | components within your application package. |
| 353 | </p> |
| 354 | <h4 id="InstrumentationTestCase">InstrumentationTestCase</h4> |
| 355 | <p> |
| 356 | If you want to use instrumentation methods in a test case class, you must use |
| 357 | {@link android.test.InstrumentationTestCase} or one of its subclasses. The |
| 358 | {@link android.app.Activity} test cases extend this base class with other functionality that |
| 359 | assists in Activity testing. |
| 360 | </p> |
| 361 | |
| 362 | <h3 id="AssertionClasses">Assertion classes</h3> |
| 363 | <p> |
| 364 | Because Android test case classes extend JUnit, you can use assertion methods to display the |
| 365 | results of tests. An assertion method compares an actual value returned by a test to an |
| 366 | expected value, and throws an AssertionException if the comparison test fails. Using assertions |
| 367 | is more convenient than doing logging, and provides better test performance. |
| 368 | </p> |
| 369 | <p> |
| 370 | Besides the JUnit {@link junit.framework.Assert} class methods, the testing API also provides |
| 371 | the {@link android.test.MoreAsserts} and {@link android.test.ViewAsserts} classes: |
| 372 | </p> |
| 373 | <ul> |
| 374 | <li> |
| 375 | {@link android.test.MoreAsserts} contains more powerful assertions such as |
| 376 | {@link android.test.MoreAsserts#assertContainsRegex}, which does regular expression |
| 377 | matching. |
| 378 | </li> |
| 379 | <li> |
| 380 | {@link android.test.ViewAsserts} contains useful assertions about Views. For example |
| 381 | it contains {@link android.test.ViewAsserts#assertHasScreenCoordinates} that tests if a View |
| 382 | has a particular X and Y position on the visible screen. These asserts simplify testing of |
| 383 | geometry and alignment in the UI. |
| 384 | </li> |
| 385 | </ul> |
| 386 | <h3 id="MockObjectClasses">Mock object classes</h3> |
| 387 | <p> |
| 388 | To facilitate dependency injection in testing, Android provides classes that create mock system |
| 389 | objects such as {@link android.content.Context} objects, |
| 390 | {@link android.content.ContentProvider} objects, {@link android.content.ContentResolver} |
| 391 | objects, and {@link android.app.Service} objects. Some test cases also provide mock |
| 392 | {@link android.content.Intent} objects. You use these mocks both to isolate tests |
| 393 | from the rest of the system and to facilitate dependency injection for testing. These classes |
| 394 | are found in the packages {@link android.test} and {@link android.test.mock}. |
| 395 | </p> |
| 396 | <p> |
| 397 | Mock objects isolate tests from a running system by stubbing out or overriding |
| 398 | normal operations. For example, a {@link android.test.mock.MockContentResolver} |
| 399 | replaces the normal resolver framework with its own local framework, which is isolated |
| 400 | from the rest of the system. MockContentResolver also stubs out the |
| 401 | {@link android.content.ContentResolver#notifyChange(Uri, ContentObserver, boolean)} method |
| 402 | so that observer objects outside the test environment are not accidentally triggered. |
| 403 | </p> |
| 404 | <p> |
| 405 | Mock object classes also facilitate dependency injection by providing a subclass of the |
| 406 | normal object that is non-functional except for overrides you define. For example, the |
| 407 | {@link android.test.mock.MockResources} object provides a subclass of |
| 408 | {@link android.content.res.Resources} in which all the methods throw Exceptions when called. |
| 409 | To use it, you override only those methods that must provide information. |
| 410 | </p> |
| 411 | <p> |
| 412 | These are the mock object classes available in Android: |
| 413 | </p> |
| 414 | <h4 id="SimpleMocks">Simple mock object classes</h4> |
| 415 | <p> |
| 416 | {@link android.test.mock.MockApplication}, {@link android.test.mock.MockContext}, |
| 417 | {@link android.test.mock.MockContentProvider}, {@link android.test.mock.MockCursor}, |
| 418 | {@link android.test.mock.MockDialogInterface}, {@link android.test.mock.MockPackageManager}, and |
| 419 | {@link android.test.mock.MockResources} provide a simple and useful mock strategy. They are |
| 420 | stubbed-out versions of the corresponding system object class, and all of their methods throw an |
| 421 | {@link java.lang.UnsupportedOperationException} exception if called. To use them, you override |
| 422 | the methods you need in order to provide mock dependencies. |
| 423 | </p> |
| 424 | <p class="Note"><strong>Note:</strong> |
| 425 | {@link android.test.mock.MockContentProvider} |
| 426 | and {@link android.test.mock.MockCursor} are new as of API level 8. |
| 427 | </p> |
| 428 | <h4 id="ResolverMocks">Resolver mock objects</h4> |
| 429 | <p> |
| 430 | {@link android.test.mock.MockContentResolver} provides isolated testing of content providers by |
| 431 | masking out the normal system resolver framework. Instead of looking in the system to find a |
| 432 | content provider given an authority string, MockContentResolver uses its own internal table. You |
| 433 | must explicitly add providers to this table using |
| 434 | {@link android.test.mock.MockContentResolver#addProvider(String,ContentProvider)}. |
| 435 | </p> |
| 436 | <p> |
| 437 | With this feature, you can associate a mock content provider with an authority. You can create |
| 438 | an instance of a real provider but use test data in it. You can even set the provider for an |
| 439 | authority to <code>null</code>. In effect, a MockContentResolver object isolates your test |
| 440 | from providers that contain real data. You can control the |
| 441 | function of the provider, and you can prevent your test from affecting real data. |
| 442 | </p> |
| 443 | <h3 id="ContextMocks">Contexts for testing</h3> |
| 444 | <p> |
| 445 | Android provides two Context classes that are useful for testing: |
| 446 | </p> |
| 447 | <ul> |
| 448 | <li> |
| 449 | {@link android.test.IsolatedContext} provides an isolated {@link android.content.Context}, |
| 450 | File, directory, and database operations that use this Context take place in a test area. |
| 451 | Though its functionality is limited, this Context has enough stub code to respond to |
| 452 | system calls. |
| 453 | <p> |
| 454 | This class allows you to test an application's data operations without affecting real |
| 455 | data that may be present on the device. |
| 456 | </p> |
| 457 | </li> |
| 458 | <li> |
| 459 | {@link android.test.RenamingDelegatingContext} provides a Context in which |
| 460 | most functions are handled by an existing {@link android.content.Context}, but |
| 461 | file and database operations are handled by a {@link android.test.IsolatedContext}. |
| 462 | The isolated part uses a test directory and creates special file and directory names. |
| 463 | You can control the naming yourself, or let the constructor determine it automatically. |
| 464 | <p> |
| 465 | This object provides a quick way to set up an isolated area for data operations, |
| 466 | while keeping normal functionality for all other Context operations. |
| 467 | </p> |
| 468 | </li> |
| 469 | </ul> |
| 470 | <h2 id="InstrumentationTestRunner">Running Tests</h2> |
| 471 | <p> |
| 472 | Test cases are run by a test runner class that loads the test case class, set ups, |
| 473 | runs, and tears down each test. An Android test runner must also be instrumented, so that |
| 474 | the system utility for starting applications can control how the test package |
| 475 | loads test cases and the application under test. You tell the Android platform |
| 476 | which instrumented test runner to use by setting a value in the test package's manifest file. |
| 477 | </p> |
| 478 | <p> |
| 479 | {@link android.test.InstrumentationTestRunner} is the primary Android test runner class. It |
| 480 | extends the JUnit test runner framework and is also instrumented. It can run any of the test |
| 481 | case classes provided by Android and supports all possible types of testing. |
| 482 | </p> |
| 483 | <p> |
| 484 | You specify <code>InstrumentationTestRunner</code> or a subclass in your test package's |
| 485 | manifest file, in the |
| 486 | <code><a href="{@docRoot}guide/topics/manifest/instrumentation-element.html"><instrumentation></a></code> |
| 487 | element. Also, <code>InstrumentationTestRunner</code> code resides |
| 488 | in the shared library <code>android.test.runner</code>, which is not normally linked to |
| 489 | Android code. To include it, you must specify it in a |
| 490 | <code><a href="{@docRoot}guide/topics/manifest/uses-library-element.html"><uses-library></a></code> |
| 491 | element. You do not have to set up these elements yourself. Both Eclipse with ADT and the |
| 492 | <code>android</code> command-line tool construct them automatically and add them to your |
| 493 | test package's manifest file. |
| 494 | </p> |
| 495 | <p class="Note"> |
| 496 | <strong>Note:</strong> If you use a test runner other than |
| 497 | <code>InstrumentationTestRunner</code>, you must change the <instrumentation> |
| 498 | element to point to the class you want to use. |
| 499 | </p> |
| 500 | <p> |
| 501 | To run {@link android.test.InstrumentationTestRunner}, you use internal system classes called by |
| 502 | Android tools. When you run a test in Eclipse with ADT, the classes are called automatically. |
| 503 | When you run a test from the command line, you run these classes with |
| 504 | <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge (adb)</a>. |
| 505 | </p> |
| 506 | <p> |
| 507 | The system classes load and start the test package, kill any processes that |
| 508 | are running an instance of the application under test, and then load a new instance of the |
| 509 | application under test. They then pass control to |
| 510 | {@link android.test.InstrumentationTestRunner}, which runs |
| 511 | each test case class in the test package. You can also control which test cases and |
| 512 | methods are run using settings in Eclipse with ADT, or using flags with the command-line tools. |
| 513 | </p> |
| 514 | <p> |
| 515 | Neither the system classes nor {@link android.test.InstrumentationTestRunner} run |
| 516 | the application under test. Instead, the test case does this directly. It either calls methods |
| 517 | in the application under test, or it calls its own methods that trigger lifecycle events in |
| 518 | the application under test. The application is under the complete control of the test case, |
| 519 | which allows it to set up the test environment (the test fixture) before running a test. This |
| 520 | is demonstrated in the previous <a href="#ActivitySnippet">code snippet</a> that tests an |
| 521 | Activity that displays a Spinner widget. |
| 522 | </p> |
| 523 | <p> |
| 524 | To learn more about running tests, please read the topics |
| 525 | <a href="{@docRoot}tools/testing/testing_eclipse.html"> |
| 526 | Testing from Eclipse with ADT</a> or |
| 527 | <a href="{@docRoot}tools/testing/testing_otheride.html"> |
| 528 | Testing from Other IDEs</a>. |
| 529 | </p> |
| 530 | <h2 id="TestResults">Seeing Test Results</h2> |
| 531 | <p> |
| 532 | The Android testing framework returns test results back to the tool that started the test. |
| 533 | If you run a test in Eclipse with ADT, the results are displayed in a new JUnit view pane. If |
| 534 | you run a test from the command line, the results are displayed in <code>STDOUT</code>. In |
| 535 | both cases, you see a test summary that displays the name of each test case and method that |
| 536 | was run. You also see all the assertion failures that occurred. These include pointers to the |
| 537 | line in the test code where the failure occurred. Assertion failures also list the expected |
| 538 | value and actual value. |
| 539 | </p> |
| 540 | <p> |
| 541 | The test results have a format that is specific to the IDE that you are using. The test |
| 542 | results format for Eclipse with ADT is described in |
| 543 | <a href="{@docRoot}tools/testing/testing_eclipse.html#RunTestEclipse"> |
| 544 | Testing from Eclipse with ADT</a>. The test results format for tests run from the |
| 545 | command line is described in |
| 546 | <a href="{@docRoot}tools/testing/testing_otheride.html#RunTestsCommand"> |
| 547 | Testing from Other IDEs</a>. |
| 548 | </p> |
| 549 | <h2 id="Monkeys">monkey and monkeyrunner</h2> |
| 550 | <p> |
| 551 | The SDK provides two tools for functional-level application testing: |
| 552 | </p> |
| 553 | <ul> |
| 554 | <li> |
| 555 | The <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>, |
| 556 | usually called "monkey", is a command-line tool that sends pseudo-random streams of |
| 557 | keystrokes, touches, and gestures to a device. You run it with the |
| 558 | <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> (adb) tool. |
| 559 | You use it to stress-test your application and report back errors that are encountered. |
| 560 | You can repeat a stream of events by running the tool each time with the same random |
| 561 | number seed. |
| 562 | </li> |
| 563 | <li> |
| 564 | The <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a> tool |
| 565 | is an API and execution environment for test programs written in Python. The API |
| 566 | includes functions for connecting to a device, installing and uninstalling packages, |
| 567 | taking screenshots, comparing two images, and running a test package against an |
| 568 | application. Using the API, you can write a wide range of large, powerful, and complex |
| 569 | tests. You run programs that use the API with the <code>monkeyrunner</code> command-line |
| 570 | tool. |
| 571 | </li> |
| 572 | </ul> |
| 573 | <h2 id="PackageNames">Working With Package names</h2> |
| 574 | <p> |
| 575 | In the test environment, you work with both Android application package names and |
| 576 | Java package identifiers. Both use the same naming format, but they represent substantially |
| 577 | different entities. You need to know the difference to set up your tests correctly. |
| 578 | </p> |
| 579 | <p> |
| 580 | An Android package name is a unique system name for a <code>.apk</code> file, set by the |
| 581 | "android:package" attribute of the <manifest> element in the package's |
| 582 | manifest. The Android package name of your test package must be different from the |
| 583 | Android package name of the application under test. By default, Android tools create the |
| 584 | test package name by appending ".test" to the package name of the application under test. |
| 585 | </p> |
| 586 | <p> |
| 587 | The test package also uses an Android package name to target the application package it |
| 588 | tests. This is set in the "android:targetPackage" attribute of the |
| 589 | <instrumentation> element in the test package's manifest. |
| 590 | </p> |
| 591 | <p> |
| 592 | A Java package identifier applies to a source file. This package name reflects the directory |
| 593 | path of the source file. It also affects the visibility of classes and members to each other. |
| 594 | </p> |
| 595 | <p> |
| 596 | Android tools that create test projects set up an Android test package name for you. |
| 597 | From your input, the tools set up the test package name and the target package name for the |
| 598 | application under test. For these tools to work, the application project must already exist. |
| 599 | </p> |
| 600 | <p> |
| 601 | By default, these tools set the Java package identifier for the test class to be the same |
| 602 | as the Android package identifier. You may want to change this if you want to expose |
| 603 | members in the application under test by giving them package visibility. If you do this, |
| 604 | change only the Java package identifier, not the Android package names, and change only the |
| 605 | test case source files. Do not change the Java package name of the generated |
| 606 | <code>R.java</code> class in your test package, because it will then conflict with the |
| 607 | <code>R.java</code> class in the application under test. Do not change the Android package name |
| 608 | of your test package to be the same as the application it tests, because then their names |
| 609 | will no longer be unique in the system. |
| 610 | </p> |
| 611 | <h2 id="WhatToTest">What to Test</h2> |
| 612 | <p> |
| 613 | The topic <a href="{@docRoot}tools/testing/what_to_test.html">What To Test</a> |
| 614 | describes the key functionality you should test in an Android application, and the key |
| 615 | situations that might affect that functionality. |
| 616 | </p> |
| 617 | <p> |
| 618 | Most unit testing is specific to the Android component you are testing. |
| 619 | The topics <a href="{@docRoot}tools/testing/activity_testing.html">Activity Testing</a>, |
| 620 | <a href="{@docRoot}tools/testing/contentprovider_testing.html"> |
| 621 | Content Provider Testing</a>, and <a href="{@docRoot}tools/testing/service_testing.html"> |
| 622 | Service Testing</a> each have a section entitled "What To Test" that lists possible testing |
| 623 | areas. |
| 624 | </p> |
| 625 | <p> |
| 626 | When possible, you should run these tests on an actual device. If this is not possible, you can |
| 627 | use the <a href="{@docRoot}tools/devices/emulator.html">Android Emulator</a> with |
| 628 | Android Virtual Devices configured for the hardware, screens, and versions you want to test. |
| 629 | </p> |
| 630 | <h2 id="NextSteps">Next Steps</h2> |
| 631 | <p> |
| 632 | To learn how to set up and run tests in Eclipse, please refer to |
| 633 | <a href="{@docRoot}tools/testing/testing_eclipse.html">Testing from Eclipse with ADT</a>. |
| 634 | If you're not working in Eclipse, refer to |
| 635 | <a href="{@docRoot}tools/testing/testing_otheride.html">Testing from Other IDEs</a>. |
| 636 | </p> |
| 637 | <p> |
| 638 | If you want a step-by-step introduction to Android testing, try the |
| 639 | <a href="{@docRoot}tools/testing/activity_test.html">Activity Testing Tutorial</a>. |
| 640 | </p> |