When I first considered getting UAT feedback early in the life of a project, it seemed like a great idea in all ways. As frequently happens, though, “seemed” is the key word. There are unintended consequences that I now know to expect on future projects that include both behavioral and process impacts.
Behaviorally, I have noticed that conditioning from non-agile projects seems to be hard to shake for those who are new to this style of project. Normally, by the time UAT comes around for a non-agile project, the time available for fixes has become quite limited and prioratization sessions can get almost cutthroat. Users want changes to the interface that will make their lives easier while programmers may be scrambling just to get all the needed functionality working correctly enough to satisfy release requirements. The project manager is all primed to cry “change control” for anything resembling material effort since time is in very short supply and critical trade-offs loom. These reactions and biases are quite reasonable if UAT only occurs at the end of the project, right before implementation. However, when the first round of UAT happens 6 weeks in to a 6 month project, these time constraints are different, but the emotional reactions don’t automatically adjust.
My first thought was that “They are asking for feedback to find out what the users want – of course they should adapt and incorporate things that make sense. Isn’t that what being agile is all about?” I realize that the wounds of any heated design discussions are still fresh, so any criticism is likely to sting a little worse. However, the length of time and pain involved in the original decision don’t mean that the user input is any less valid, so they should just get over it, right? If the user feedback is saying that there were erroneous assumptions, then that should be considered. Otherwise, why are we bothering to ask for their feedback? We can’t just listen to the kudos. If it turns out that the feedback is based on wrong assumptions, then we incorporate that in to training and future sessions. Case closed, right?
However, the question of time trade-offs does remain valid, even though it becomes more tricky to evaluate. It is true that time spent refactoring and changing approaches may take away time that will later be needed for functionality implementation. The problem is that it is far harder to convince people of what the exact opportunity cost is since you haven’t yet defined those later portions in enough detail to do a precise estimate. It may be a problem, or it may save a lot of time and effort later and end up a wash for time invested. Not only can you never be 100% sure what the future cost will be, it is quite likely that you will never be able to prove the exact cost after the fact either since there will be many other factors that impact the final delivered code. You can say that you just need to manage client expectations about what can be delivered, but that doesn’t make you any more able to definitively estimate the cost trade-offs. You may also be in a situation (as we are) where the end-date has been set with an outside (waterfall based) PMO group.
The other challenge is that you may decide to defer those low priority bugs for quite good reasons until you know better what the end-of-project priorities are. However, the users have given you their feedback now, and the clock is ticking while they wait to find out whether you are going to do anything about their requests. Every time they see that “bug” when they do subsequent UAT sessions for later functionality, they will be reminded that their concern still hasn’t been addressed. Even if they know that it is just a question of time, it won’t change the emotional reaction to seeing the problem again and again. They also may or may not remember which things they reported before and which they haven’t. If you have any sizable number of low priority bugs sitting out there, it will be harder to keep track of what is on the list. If those low priority bugs wouldn’t be hard to fix (such as some UI layout changes), it becomes even harder for the users to understand why they are being delayed.
The problem is definitely not intractable, so there are steps you can take to find the right balance between user requests and project schedule priorities. It just isn’t quite as simple as it looked at first blush. I hate to think that creating a “change control” process is a good idea for UAT-based requests, but there is a real risk of opportunity costs down the road that needs to be considered. This seems to be making communication skills and perception management even more critical for successful QA efforts.
