Do testers have to open the app every day?

Google looks at whether the opt-in lasted 14 days. Daily opens are not a Google requirement. That said, testers who never open it at all are hard to explain on the production questionnaire.

If one of the 12 leaves, do I start over?

No. The remaining testers keep their elapsed time; only a replacement needs their own 14 days.

A tester says the app cannot be found, but the link works for me.

That was 87 of the 105 reports we measured. In order of frequency: country not open, login account differs from the Play Store account, group not joined. Check the country first.

How many testers should I recruit to be safe?

27.6% finished all 14 days in our data. If you need 12, recruit with slack and send a reminder on day 12 or 13.

English · 한국어 · 日本語 · 简体中文 · 繁體中文

Android closed testing, measured

1,862 test runs · 8,054 daily check-ins (16 Aug – 10 Sep 2026)

Plenty of posts explain how Android closed testing works. Almost none say what happens when you actually run 12 testers for 14 days. We operate a service where developers test each other's apps, so we have that data. Here it is.

Period — 16 Aug to 10 Sep 2026 · 144 developers · 115 apps · 1,862 test runs · 8,054 daily check-ins

1. One in four testers finishes all 14 days

Counting only the 116 runs that actually passed the 14-day mark. A run still on day 5 is not a drop-out.

Finished 14 days — 32 (27.6%)
Dropped out — 78
Never opened it once — 6 (5.2%)
Average days logged — 10.3

Recruiting 12 does not mean 12 finish. Recruit with slack.

2. But people do not quit early

We checked which day each of the 78 drop-offs stopped on. We expected a rush of quitters in the first three days. It was the opposite.

Stopped on days 1–5 — 19
Days 6–9 — 9
Days 10–13 — 50

Two thirds of drop-offs happen in the last four days. These people did not lose interest. They opened the app daily for over a week and then missed one day — busy, new phone, notifications off.

So one reminder on day 12 or 13 beats ten onboarding messages. More in Why testers drop out.

3. 83% of "I can't install it" is not the link

Testers filed 105 reports saying an opt-in link would not open. Each time we checked automatically whether anyone else had entered that same app in the previous 7 days.

In 87 of the 105, other people were getting in fine. The link worked; that one person could not use it. Three causes covered nearly all of them.

1. The country is not open — 61 reports

The most common by far. If the closed test's country list is narrow, testers elsewhere cannot even see the app in the store. They get This app is not available in your country and nothing else. The countries the reports came from:

GB 13 · KR 12 · IN 6 · BR 6 · SE 6 · JP 5 · FR 4 · US 4

Open the country list before you recruit — Play Console → closed testing → Countries / regions. See When a country blocks the install.

2. Their login is not their Play Store account

Joining the group and opting in only work from the Google account signed into the Play Store on the phone. Anyone with several Google accounts splits here.

When you ask a tester for an email, ask for "the account signed into your Play Store", not "your email". Same question, different answer.

3. They hit the opt-in link before joining the group

With a Google Group the order matters: join the group → opt in → install. Opting in first shows You are not able to join this program, and that screen never says "join the group first". The tester concludes the link is broken. Screens and order are in Connecting a Google Group.

4. And it takes time to propagate

Opting in does not put the app in the store immediately — it can take about 30 minutes. Testers who do not know this close the page and give up. Tell them up front.

Checklist before recruiting

Closed test open to all countries?
Asked for the Play Store account, in those words?
If using a group, explained join → opt in → install?
Warned them about the ~30 minute delay?
Recruited more than 12? (27.6% finish)
Reminder ready for day 12–13?

The first four alone remove most "it will not install" messages.

Frequently asked

Do testers have to open the app every day?: Google looks at whether the opt-in lasted 14 days. Daily opens are not a Google requirement. That said, testers who never open it at all are hard to explain on the production questionnaire.
If one of the 12 leaves, do I start over?: No. The remaining testers keep their elapsed time; only a replacement needs their own 14 days.
A tester says the app cannot be found, but the link works for me.: That was 87 of the 105 reports we measured. In order of frequency: country not open, login account differs from the Play Store account, group not joined. Check the country first.
How many testers should I recruit to be safe?: 27.6% finished all 14 days in our data. If you need 12, recruit with slack and send a reminder on day 12 or 13.

These numbers come from ACT Party, where developers trade closed tests with each other and daily opens are recorded automatically from Android usage stats. See the board · Get the app