The design of an effective website requires the designer to consider multiple factors (such as the screen resolution, size, readability, and navigability of the website. Initially, web designers focused on the development of websites that would meet the needs of computer users. However with the advancements of technologies many people are using other devices to access websites, such as tablets and mobile phones. This necessitates the design of websites that are accessible by web users, in respect of the device they are using. A responsive website is the best solution to these challenges.
Testing a Responsive Website
Although a responsive website has several benefits to the organization and the end users, it presents numerous challenges to the software tester. These challenges include a unique testing environment, lack of automation framework, and misleading design.
Challenge 1: Testing Environment
First, the testing environment presents some challenges because of the plethora of operating systems, browsers, platforms, and mobile devices that are required to replicate the end users experience. For example, a website on a desktop monitor with 1600×900 pixels will render differently when scaled down to an iPhone 5’s 4-inch Retina 1136-by-640 pixel screen, which will also be different on a Galaxy S4’s five inch screen with 1080×1920 pixel resolution. Ensuring that mobile websites will work on all types of devices (smartphones, tablets, and laptops) produced by the major brands (Apple, HTC, Nokia, Samsung, Sony, etc.) and on all the platforms (Android, BlackBerry, iOS, Windows, etc.) is a daunting task. Additionally, new devices are constantly being introduced into the market, so there’s an almost constant game of catch up in the quest to cover all the major devices. This means that the tester should test the functionality of the website on the most popular devices and platforms.
It is not feasible to test on each and every available device/OS/browser combination, which means testers have to strategically choose multiple configurations of physical devices. Tracers like Google Analytics can track which browsers, operating systems, and devices that the majority of people are using to reach your site. This information helps narrow your testing scope. It is important to create something on the lines of graded browser support used by Yahoo to ensure that major platforms are covered. Using emulators is another method to overcome some of the issues of testing on numerous devices.
Challenge 2: Lack of Automation Framework
Secondly, not having an automation framework will hamper the efficiency and thoroughness of the testing efforts. An automation framework is an integrated system that establishes the rules of automation testing of the product. The framework offers the basis for and simplifies test automation. Frequent application changes can make automation a huge aid in completing responsive testing across multiple platforms. However, without an automation framework that has specific verification points, the test process becomes difficult and time consuming, making smoke and regression testing timely, and cost ineffective to automate. As a result, you cannot perform automation testing on responsive sites and manual testing time must be increased to handle all the smoke and regression testing.
Here at Segue Technologies we use Selenium WebDriver, an automation tool, to tackle many of these changes. Selenium WebDriver can be used in conjunction with other tools to increase test coverage and the variety of mobile devices we can test on. Webdriver scripts can run on real devices and in an Android emulator or in the iOS Simulator. Sauce LABS gives us the ability to run their tests across hundreds of real browsers and platforms. Segue has used Selenium WebDriver with Sauce LABS for the testing of its own website. Sauce LABS takes screen shots after each page-altering command (issued by the selenium script) and a person can flip through the shots to diagnose errors and verify layout. Or, they can also watch a screencast of the whole test from start to finish.
Challenge 3: Misleading Design
Third, design strategies and approaches can easily mislead the tester, especially if they are not familiar with what to test and how they expect to test it on several devices and across different size break points. The nature of a responsive design is for the screen to display differently and appropriately to each screen size, which creates multiple versions for the tester to verify at each. Interactions that are standard on a desktop device may be irrelevant on a mobile device, and vice versa, because we interact with desktop and mobile devices differently. We use a mouse and keyboard shortcuts to navigate on a desktop, but touch and gestures are used on mobile devices, and not all interactions transfer over from one device to another. For instance, the designer may build in links that are displayed with a hover-over-effect in the desktop version. How is that feature transferred into a touch-screen mobile version where there is no mouse? Most touchscreen devices have gestures that are not possible with a desktop setup, unless the user has a touchscreen monitor. The different behaviors and the challenge to simultaneously accommodate for both regular and touch screens requires some excessive testing. Knowing the interaction methods of the different platforms shapes the testing approach and strategy. A multi-platform tester will learn to recognize which test cases do not apply or must be edited when applied to various platforms. Without this knowledge, test cases can be selected and applied incorrectly, resulting in inadequate coverage on some platforms and false test run failures.
A responsive website offers a suitable solution to the key challenges arising from the increase in the use of mobile devices to access the internet. This is because the fluidity of the responsive website allows the content to fit on screens of any size. Most importantly, a responsive website saves time and money that would be required to design different websites for various devices. However, responsive websites can make testing efforts much more difficult than for a non-responsive site. These challenges may be reflected in development costs and timelines, but they are worth it to ensure a positive user experience and proper site functionality for all of your visitors, regardless of the device they are using.
Originally published January 21, 2014.