Very large data for Xcode iOS unit tests
Summary How do I get realistic automated unit tests for multiple large data sets for an iOS application? The more realistic the test, the harder it is to do; and using a continuous-integration server (the only practical way to conduct tests) compounds the problem. But people do it. How? I make a number of flat assertions. Any may be incorrect. Correcting them will go a long way toward answering my question. Project This is an iOS 9 application that analyzes large video files (full-resolution, maximum frame rate, 10 seconds typical) taken on an iOS device. (Part of the analysis will go through OpenCV, if that matters.) The client has given us a proof-of-concept (Python on the desktop) and hundreds of sample files. Call it 8 GB total. We’d like to unit-test our code against as many of these as possible. My problem is, how do we perform automated unit tests against large data sets for an iOS target? Alternatives The better the test, the harder it is to perform: We’ll start with an OS X application to check correctness and relative speed. It’s easy to point the tool at any number of files. However, x86_64 has a different ABI, OS X a different API, and iOS has resource constraints the Mac won’t reproduce. This affects both correctness and performance measures: You could go on wild goose chases for optimizations that work on the desktop but not the device. We will eventually have to use the Accelerate framework, which can’t possibly perform the same on the two platforms. After porting to iOS, we can use the iOS Simulator, which even uses i386 (and therefore a 32-bit word) for older targets. (I explode with delight to discover that the precision of CGFloat varies between platforms.) Still the problems of ABI, performance, and constraints, compounded by the fact that the Simulator’s “iOS” API often calls through to the approximate equivalents in OS X. Additionally, I don’t believe the simulated device can break the sandbox to read files outside the device’s tree. We must (am I right?) copy the test data into the storage of each simulated device, at paths that will differ for each developer and are not guaranteed to be stable across versions of Xcode. If that were solved, even a big machine would still have problems with n × m GB. (We could link all the “stored” files to a common copy. The reliability and stability issues delight me even more.) Eventually we have to test on a device, because we need unpaged memory, storage that is a small multiple of the file size, on a throttled ARM processor that changes significantly every year. Which means pushing video files through the tether, possibly seriatim. It’s unrealistic to challenge the device with hundreds (or dozens) of files, but an adequate test suite might demand 8 — enough to be cumbersome. I’m not sanguine about there being a straightforward way to load and unload a series of files into a tethered device. I guess we might have separate test targets, each with two or three files stashed into the application data package for the Test scheme of each. I’d rather not if there’s a cleaner way. Integration It gets worse: Testing will be a long, demanding process, multiplied by each targeted device. It is not realistic to ask any of our developers to host the tests (everybody is BYOD and occasionally off-site, let’s not argue about it). Nor is it realistic to commit the test files to a shared repository. (GitHub flatly refuses.) That’s true no matter how you perform the tests. This points to builds and tests on a continuous-integration server. We wouldn’t have to bear the tedium, but the practical problems of keeping a library of test data and loading it into the sandbox of the device (simulated or practical) don’t go away. Question I know this is a solved problem. Large-file projects for mobile devices exist. The developers test them. How is it done?
iOS URL Scheme for different targets
How to place the buttons in UIStackView next to other based on condition?
Token Issue with Firebase
iTunes Connect automatically generating app ID's
Different versions of XCode and splash screen laying in Resources vs Images.xcassets
Why button not working after subview Swift view into objective-c view?
Next VC is not showing
Adding a Back Button on A Table View Controller
ionic Cordova Failed recording audio on IOS using Media plugin
Image on annotation Swift
How to show the Google Photo Sphere on an iOS app?
swift closure pass by reference?
Customize URL for iOS sharing
multiple to-many keys not allowed here
FileManager and urlsForDirectory Error in Swift 3 Xcode 8
self.removeFromSuperview() within UIView subclass