Reflection #2 – Formative vs. Summative Testing

This week our assignment was to, using the medium of our choice, convince the CEO and CTO of a fictional pizza delivery company what type of usability method they should use as they build a new website before Super Bowl Sunday. Basically, should Papa John and Dom Inos decide to do multiple tests as they develop their site and before they release it to the public? Or, should they opt to do a study at the end of development/after the site has been released to the world? Any testing is better than no testing, but… I would prefer the formative approach.

Why? Because the ability to iterate on both the research and development portions of the site will help to better match a user’s goals/wants/needs. If you wait until the end of the whole process to test, and you are up against a deadline for release, it’s going to be fairly difficult to make massive changes. However, formative tests under a tight schedule will require an agile pace for everyone involved.

In my current role, we try to do as much agile, formative testing we can. The problem comes when we are not informed of research needs in advance; many times a project team will approach us to do a usability study at the last-minute. So, a formative test has now become a summative test. Any changes that need to be made based on the findings are either prioritized for a later release, or are de-prioritized and forgotten until someone takes up the cause again. But positioning research and UX early in the product life cycle is probably best reserved for another post.