bytedance/Fastbot_iOS

Introduction

Fastbot is a model-based testing tool for modeling GUI transitions to discover app stability problems. It combines machine learning and reinforcement learning techniques to assist discovery in a more intelligent way.

Related: Fastbot_Android

***More detail see at Fastbot architecture

update 2022.1

• update Fastbot Revised License
• release AnyTrace, the Fastbot test management assistant: supports one-click Fastbot test start, crash analysis, etc. (AnyTrace User Manual)

Prepare test environment

• cd Fastbot-iOS && pod install --repo-update
• Open Fastbot-iOS.xcworkspace, Set FastbotRunner Signing & Capabilities and Bundle ID
• USB connected the device & trust the device, if you're using a simulator, start up the simulator
• Open FastbotRunner network permission (unnecessary for simulator), a sample on device 00008030-001054A80C82802E：
• • Open XcodeIDE run testPingNetwork in Tests or Run command:

BUNDLEID=com.apple.Pages duration=240 throttle=300 xcodebuild test  -workspace Fastbot-iOS.xcworkspace -scheme FastbotRunner  -configuration Release  -destination 'platform=iOS,id=00008030-001804563E44802E' -only-testing:FastbotRunner/FastbotRunner/testPingNetwork

• • By tapping FastbotRunner on the device, the screen of the device would go black for about one minute. During the black screen interval, users should press the home button on the device to go back to the main screen. Wait patiently until the network setting dialog window pops up. Users should allow the pop up request in order to continue.
• • If "ping network success" appears in the console log, that means get network permission successful

Run Test

• Ensure that your application can run on the device. (Installed and trusted)
• Environment Variables should be setted in command line or Xcode IDE/Scheme/Test

• A sample run test on device 00008030-001054A80C82802E. if IDE scheme Env Vars changed , command Env Var would be void:

BUNDLEID=com.apple.Pages duration=240 throttle=300 xcodebuild test  -workspace Fastbot-iOS.xcworkspace -scheme FastbotRunner  -configuration Release  -destination 'platform=iOS,id=00008030-001804563E44802E' -only-testing:FastbotRunner/FastbotRunner/testFastbot

More detail see at 中文手册

Advanced Extension

Stub mode: Target dynamic library fastbot_stub. Stub mode requires injection of fastbot_stub into the test app. The library captures GUI structure by parsing the app under test for fastbot. More customized features (eg. hook callback, cut View) can be constructed by users for additional abilities such as blocking certain view from being clicked, customized ViewControllers, etc.

We highly appreciate any contribution from the community !!!

Usage: After injecting fastbot_stub to app, you need：

• Uncomment code block [fastbot_native addUIInterruptionMonitor: ...]; in FastbotRunner.m
• Run the stub mode by editing Scheme Environment Variables(9797 can be changed to another port number):

Analytics

To prioritize and improve Fastbot-iOS, FastbotRunner collects usage data and uploads it to Google Analytics. FastbotRunner collects the md5 hash of the test app's Bundle ID, this information allows us to measure the volume of usage. If they wish, users can choose to disable the Analytics by skip step Open FastbotRunner network permission or change FastbotRunner's Wireless Data to off in System Preference.

Support

• Public technical discussion on github is preferred.

• Q&A:

  Q: Get Error when pod install --repo-update
  A: install pod firstly sudo gem install cocoapods -v=1.8.1
  

  Q: Get Error: Assert Fail Timed out while evaluating UI query
  A: Restart test or Replug USB or Change a USB line or Restart iPhone
  

  Q: Get Error when use simulator
  A: Change to Debug Mode in scheme setting
  

  Q: Get unkown install Error:com.apple.dt.MobileDeviceErrorDomain
  A: Check your signing certificate or Replug USB or Change a USB line or Restart iPhone
  
  

License

Copyright©2021 Bytedance

Licensed under Fastbot Revised

Fastbot-iOS required some features are based on or derives from projects below:

• WebDriverAgent licensed under BSD-3-Clause

Publications

If you use our work in your research, please kindly cite us as:

1. Lv, Zhengwei, Chao Peng, Zhao Zhang, Ting Su, Kai Liu, Ping Yang (2022). “Fastbot2: Reusable Automated Model-based GUI Testing for Android Enhanced by Reinforcement Learning”. In proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE 2022). ACM, To appear. [pdf]

@inproceedings{fastbot2,
  title={Fastbot2: Reusable Automated Model-based GUI Testing for Android Enhanced by Reinforcement Learning},
  author={Lv, Zhengwei and Peng, Chao and Zhang, Zhao and Su, Ting and Liu, Kai and Yang, Ping},
  booktitle={Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE 2022)},
  year={2022}
}

2. Peng, Chao, Zhao Zhang, Zhengwei Lv, Ping Yang (2022). “MUBot: Learning to Test Large-Scale Commercial Android Apps like a Human”. In proceedings of the 38th International Conference on Software Maintenance and Evolution (ICSME 2022). IEEE, To appear. [pdf]

@inproceedings{mubot,
  title={MUBot: Learning to Test Large-Scale Commercial Android Apps like a Human},
  author={Peng, Chao and Zhang, Zhao and Lv, Zhengwei and Yang, Ping},
  booktitle={Proceedings of the 38th International Conference on Software Maintenance and Evolution (ICSME 2022)},
  year={2022}
}

3. Cai, Tianqin, Zhao Zhang, and Ping Yang. “Fastbot: A Multi-Agent Model-Based Test Generation System”. In Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test. 2020. [pdf]

@inproceedings{fastbot,
  title={Fastbot: A Multi-Agent Model-Based Test Generation System},
  author={Cai, Tianqin and Zhang, Zhao and Yang, Ping},
  booktitle={Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test},
  pages={93--96},
  year={2020}
}