Conquering the UAT Process: Before, During & After

User Acceptance Testing can be a daunting and frustrating experience. Too often, the exercise becomes an ordeal of tight deadlines, stress, and system issues. While UAT will always be a high-effort activity, good preparation, responsiveness, and follow-up will multiply your chances of success.

TWEETTweet: Conquering the #UAT Process one step at a time | #Abraic #pmot https://ctt.ec/hJReY+

BEFORE UAT

Define roles & responsibilities

Identifying the official UAT Team is the first order of business. Each team member’s role and responsibility should be clearly defined – but this doesn’t need to be a formal exercise. A 15-minute meeting works well to assign roles and responsibilities.

The UAT team should consist of two cohorts: a Solution Camp and a Testing Camp.

Solution Camp Roles:

  • UAT Lead: Team leader; ultimately responsible for UAT success.
  • Internal IT: Supports the UAT Lead with frontline IT tasks (computers, peripherals, printers, etc.).
  • Consultants/Vendors: External parties support the project with subject matter or technical expertise.
  • Support Team Liaison: Manages the support team; communicates issues and solutions to the UAT Team and Support Team.

Testing Camp Roles:

  • Business Sponsor: Energizes and motivates participants and their managers; keeps the business case in mind.
  • Product Owner(s): Responsible for knowledge of the product requirements, or what the users want. Has authority to make decisions on functionality/requirements. (This does not have to be a formal role; its responsibilities can be covered by the Business Sponsor or the Functional Leads.)
  • Users: Performs tests.
  • Business Functional Leads/Experts: Performs tests, or assists testers with processes.

Define success

Once the team is established, the Leads should work with project stakeholders to agree on a definition of victory. What does success look like? Is this the type of project that can only go live if there are zero issues, or is there some room for rework? These questions need to be answered and documented. The answers will guide UAT overall.

Set a schedule

Once the team is established, the UAT Lead should work with experts on the UAT Team to create the UAT Schedule. The general pattern for UAT is:

  1. Testing
  2. Rework
  3. Validation of the fixed issues

These three phases can be repeated as many times as required until the definition of success is met. Be sure to leave room for all the phases, as even fix validation can be time-consuming.

One more note about the testing phase: the schedule should include space between sessions, so minor delays don’t derail the entire phase. A great testing schedule will take all dependencies into account. For example, shipping transactions should not be tested before sales orders have been entered.

Initiate the communication plan

When the schedule is complete, it’s time to communicate, communicate, communicate. And in case it’s not self-evident, one email is not enough to effectively communicate your upcoming UAT.

TWEETTweet: “1 email is not enough to effectively communicate your upcoming UAT” #ITComms #Abraic https://ctt.ec/MFkeV+

The users and testers need to know about UAT well in advance of testing. They need to know the scope and purpose of the project, and what benefits are expected of the new software. The UAT Schedule will allow you to estimate time commitments required of the testers, which should also be communicated. If UAT must be scheduled during a vacation season, request PTO schedules early and adjust the plan accordingly. Be so persistent that even after-hours cleaning staff is aware of UAT. The Business Sponsor should pursue managerial buy-in. Department managers can be a great help to motivate their functional users, and to prioritize UAT. Without managerial support, attendance can become a big problem.

Prepare the users

If the testers have never seen the software they’re expected to test, the UAT team will need to provide demos or training beforehand so testers are not going in blind. If you are working with users who have already been involved in the project, then they should have some familiarity with the test scripts. But if it’s been a while, test scripts should be reviewed and updated, with assistance from functional experts as required.

In general, the more UAT participants who see the scripts in advance, the better. If the scripts are familiar before testing begins, participants will spend less time trying to understand scripts, and more time testing.

Coordinate logistics

Lastly, the non-glamorous activities should not be forgotten. Book conference rooms in advance to ensure available space. Make plans to set up the rooms with spare computers, and any necessary peripherals: printers, RFID scanners, etc. The support team will need to set up test environments well in advance and ensure participants have user accounts with appropriate security. Map the computers, peripherals, printers to the test environments before testing begins. The support team should also do a quick “smoke test” – check that applications open, data is present, logins work, security looks ok.

Establish a method for tracking progress

There are many ways to track UAT progress, and the best method will depend on your specific project. For example, some software will have clear-cut processes, with one test script per process. Other software is more complex, and progress depends on end-to-end functionality working fully. Either way, it is important to track success, fails, and outstanding tests. (These metrics can also be considered success criteria, e.g. “We will cycle through UAT phases until 95% of scripts are successful.”)

DURING UAT

Start on the same page

Initiate UAT with a kick-off session for the UAT Team. Review roles and responsibilities again so everyone is on the same page.

Combat absenteeism

Ask one person to chase down absentees. If participants do not show up to a session, call them or walk to their desk. Do not hesitate to escalate poor attendance to the Business Sponsor, who should communicate with Managers. If you were able to gain managerial support before UAT, managers will be ready to assist with their direct reports who are struggling to attend sessions.

The UAT Team should give their full participation. It becomes a problem when the Lead, Sponsor, or external experts are unavailable. It should be clear that UAT is their only job for the time being. This goes for the system experts and support team as well – UAT issues are the number one priority.

Report issues as they occur

During testing, one person should report issues to the Support Team as they are discovered. If the Support Team is online during testing, issues can be called in by phone.

Track progress in real time

Testing should be tracked against the Test Scripts, with a 10-minute progress meeting at the end of each day. If everyone is aware of progress, adjustments will be easier to make. Issue resolution progress should be reviewed as well, with meetings led by the Support Lead. Clear action items and owners are critical for fast-paced issue resolution.

Bolster morale

Keep an eye on stress levels. Take advantage of age-old morale traditions, such as bringing donuts for early morning sessions, or snacks for the afternoon. A coffee machine or electric tea kettle in the room could make testing a bit more enjoyable.

AFTER UAT

After all the UAT sessions are completed, the team still has a little more work to do…

Close out issues

Usually, there are a few last outstanding issues that didn’t get addressed in the UAT cycles. As fixes come in, the UAT Team should seek the original tester to validate the solution.

Review testing progress against scripts one last time. If anything was left untested, individual sessions can be scheduled to mop up.

Get sign-off

Probably the most important task after UAT is documenting tester approval. If your company has a responsive email culture, that may be the cleanest way to go. Email approval can even be requested by using Voting Buttons in Outlook. However, if everyone’s inbox is swamped with hundreds of unread messages, physical signatures may be more effective.

Since the UAT Team faithfully tracked testing against scripts as described above, you should know exactly who needs to sign off on which tests. Assign one or more UAT Team members to walk the aisles with clipboards and ask testers for approval. Be sure to scan the paper approvals and digitally store them in a secure place for easy access.

Be sure to communicate the plan to the testers – they should not feel like everyone will disappear as soon as they sign off and no new issues will ever get fixed again. Address the fear of abandonment by proactively explaining the next steps of the project.

Express gratitude

Lastly, it’s a good idea to send a thank-you email to the testers and their supervisors to show that their time was valued and appreciated.

TWEETTweet: Conquering the #UAT Process one step at a time | #Abraic #pmot https://ctt.ec/hJReY+


Good luck! Let us know if you use this checklist, and how it worked for you.

4 Comments

Adam Caire

Great article Andrey there’s some really good information there. How do you handle product scope changes during UAT? This has happened at my company recently and was poorly managed and we seemed out of control of this process. Your thoughts?
Regards,
Adam.

Reply
Andrey Alekseyev

Hello Adam! Thanks for the compliment.

Product scope changes as late in the game as UAT are a risk. My advice would be to raise communication and expectation management to be the top priority. Changing the scope will result in different testing requirements, which will affect the timeline and dedication needs from testers. Basically, the cascading effects may be massive – but should be possible to predict. And once the prediction is made, decision makers can be informed and make the call. If the scope changes are truly critical, then the schedule should be adjusted. If they’re not critical… maybe they can be deferred to a later date.

I’d love to hear how your situation turned out! Please send me a note at aalekseyev@abraic.com

Reply
Tara

I have a question related to how to get SME / User Engagement for cross-functional UAT? We have some large enterprise endeavors that require full end to end testing, and finding ALL resources/SME users to participate is difficult. Looking for suggestions on combating and engaging.

Reply
Andrey Alekseyev

Hi Tara, thanks for your comment! SME and user engagement is the most important part of UAT. After all, the point is for users to accept the new software or features. From my experience, a strong Business Sponsor on the Testing Team is a huge advantage for user participation. The Sponsor (or a small team if there isn’t one person in the role) needs to communicate the project goals and timeline to the user community and their managers ahead of time. And not just communicate, but collaborate on the details and logistics of the plan. If department managers help design the plan and agree to it, it’s a lot easier to ask for their help getting users into testing later on. We often deal with long cycles of end-to-end testing, and the key is to agree to a realistic plan collectively, leave enough room between sessions that minor delays aren’t catastrophic, and keep managers in the loop so they’ll be your allies on participation. Let me know at aalekseyev@abraic.com if you’d like to speak further, and good luck!

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *