Apple TestFlight: is it allowed to download 2 beta applications for A / B test?

Question

I need to run a beta test for an iOS 8 application through Apple TestFlight. Is it allowed to download 2 beta versions of the A / B test? (I would like to switch the application icon and some other things to check what works best for users.)

---

Award notes:

It is especially interesting if there is experience with the presentation of two similar assemblies for external testing, since the assemblies must go through (albeit easier).

I would suggest that everything will be all right, because you can submit free form notes for reviewers and explain the situation - and because in any case, applications that have been tested outside must go through the usual App Store review before they go live; therefore, acceptance in external testing would not be free on the App Store.

But these are only my assumptions and, therefore, generosity. Has anyone done this? With negative or positive results? Or both review processes are deliberately independent and sometimes arbitrary.

I am not interested in the interpretation of review principles , but how they are implemented in practice.

Or should all this be handled internally with some testing of the A / B framework? (Unfortunately, this is more than just creating two package identifiers ... And this will not help when testing the application icon)

Answer 1

To have 2 active assemblies in the test, you need 2 applications (different package identifiers), the only choice you have is either through the same account or using different accounts. It is easier to handle in one account, less context switching for management.

We successfully worked without any problems raised by Apple. We also did not add any special comments to the review notes.

In the end, you will get more dead projects in your account, but this is the only problem for you, Apple will not complain.

Why hard

A / B testing can be performed in the same assembly, with the exception of some things, as you mentioned (application icon). This is usually handled through surveys.

The only advantage of having two applications is that all users can test both versions in your A / B test, tempered by their will, to spend time testing.

The disadvantages that come to mind:

• sooner or later you will kill one of these versions, potentially losing some of your beta testers who have chosen only this version. It softens the fact that you will not need beta testers after the release of the version of Appstore
• you won’t be able to control who has which version to ensure proper distribution and transfer of users from one to another if you are not ready for a lot of hands when communicating with your beta testers.

Answer 2

I could not have multiple versions in TestFlight. If I did, I would use a separate initialization file and an iTunes Connect account. There are several steps to change the initialization account and application name so that it is unique, but I think this is the only way around it. However, to use external testers (since internal testers are limited to 25), you will have to go through the review process. Apple may object to having to repeat the same application twice if it catches it. I would advise you to re-read the Terms and Conditions , remembering that you are sending a request for a beta version of the application.

Answer 3

As of April 11, 2017, Testflight allows you to distribute and test multiple assemblies at the same time.

https://developer.apple.com/news/?id=04112017a

Answer 4

You can download multiple assemblies, but you can only test ONE at a time, and you cannot do what you want, except that you create a new unique application identifier and load the same application into the new APP application identifier.

Answer 5

You can download more than one assembly of the same version !!! Just change the build number. Example: 'version 1.0, build 1.0', 'version 1.0, build 1.1'. You can see different assemblies in test flight.