Testing practicalities depend on the
project size and objectives. Unfortunately, projects quite often do not
always concentrate on these topics enough during the development phase.
A key indicator symptomatic of not understanding the value of testing
is the time your project has reserved for it. This is a classic mistake
in research and development, as well as in customer-oriented projects,
which often delays the deployment to your production environments.
Setting Up a Testing Strategy
Set out a test strategy, or plan for
your test manager to produce a test strategy that describes the
approach that will be used to test the solution and the areas that will
be tested. The test strategy should cover the following areas:
- Human Resource — How many resources are required to test the solution?
- Hardware requirements and environments
— This covers the hardware requirements and environments required to
adequately test your SharePoint 2013 platform and solution.
- Software requirements and specialist testing tools — This covers what software is required for the hardware and the software testing tools needed to adequately test the solution.
- Test case creation process — This covers what test cases will be written and the steps required for each test.
- Types of functional and nonfunctional tests
— This covers how common SharePoint 2013 customizations will be tested,
as well as the nonfunctional tests that will be performed. For example,
this may include tests related to accessibility, localization,
security, compatibility, performance, maintainability, and data
migration.
- Testing stages — This should cover the unit, component, integration tests, system, regression, and user acceptance tests.
- Defect tracking — Defect tracking
specifies what tool will be used to track defects. In
Microsoft-orientated projects, Team Foundation Server (TFS) is often
used as the tool to track defects.
- Defect classification strategy —
The defect classification strategy ensures each defect is uniformly
categorized and prioritized based on an agreed set of severity
definitions. Examples include 1-Critical, 2-Major, 3-Minor, and
4-Trivial. This is important because customers often stipulate test
exit criteria in the commercial contract that specify the maximum
number of defects allowed for each severity definition.
- Triage strategy and triage process
— This process is used to classify and prioritize defects. This meeting
occurs regularly during the development stages of the project and daily
during the stabilization phase near the end of a milestone.
One final area to plan is the test team roles and
responsibilities; that is, the responsibilities of the test lead and
the test team members.
Unit Testing
Unit testing
is used to ensure that individual units of source code work as
expected. Unit testing can be performed either manually or using
automation by writing code-based tests to verify the code works
properly.
For unit testing, one of the challenges in
SharePoint development is the nature of SharePoint code and how
developers implement logic in the user interface (UI) classes (such a
web part or custom control). This makes it difficult to test business
logic residing in the UI class, and this makes it even more difficult
to ensure that your automated unit tests cover the majority of your
code base.
If your customizations implement well-known
patterns such as model-view-controller (MVC) or Model-View-ViewModel
(MVVM) patterns, this can make it easier to test the logic of your UI
classes.
Automated Builds and Integration Testing
Automated builds are used to automate
integration testing of the customizations using a daily schedule, or
whenever a developer checks in code. The objective is to ensure that
code created by your development team integrates without any issues.
This prevents large integration phases where overlapping code requires
fixing and rework, and enables you to take a build at any time to
deploy to other environments (such as your test environment).
Often, automated builds are extended to include
other activities, such as unit testing or automated functional testing.
By scheduling these to be actioned automatically, you can save on
overall costs concerning quality assurance (QA). You can provide a new
fully working version of the latest checked-in developer customizations
in your test environment daily. When the customer or the tester arrives
at work in the morning, results and indications of the expected test
result will be available to analyze.
For example, say that you have already deployed a
release of your solution to your customer (internal or external), and
you want to ensure that the following iterations or maintenance builds
could be easily deployed or actually upgraded over the existing
environment.
In this kind of scenario, you can use the model defined in Figure 1.
Here, you test the upgrade actions that must be performed as part of
your automated build. Combining virtualization techniques, you mimic
your production environment against your daily build. In a best-case
scenario, you would have a legacy copy of your customer content
databases from the production environment to perform this daily test.
Understandably, it is not always possible to have
a copy of production data in your test environments. You can still
complete the following steps in Figure 1:
1. Perform automated unit testing
— Before moving to the following steps, verify that the customizations
actually work as expected. To verify business logic handling of the
solutions, you create unit tests, which are automatically executed as
part of your automated build. This way, any business-level issue can be
identified as early as possible in your development cycle without
requiring manual testing.
2. Compile new package
— The automated build compiles the code stored in your source control
system (optimally based on labeled status) to enable you to use the
latest stable build, rather than simply the latest source code version.
This avoids code that is in flux and is labeled as not ready to be
released. This requires minor tweaks to your automated build
configuration files (MSBuild scripts). This helps ensure that only
correct source code versions are used.
3. Restore snapshot
— Because automated builds are executed on a daily basis, you must
restore your test environment to a base configuration that mimics the
state of your current production environment, ready for your new build.
Depending on your virtualization platform, this could be difficult to
establish. With Microsoft Hyper-V, you can perform this action easily
(using Windows PowerShell, for example) so that it can be easily
automated and included in the build process.
4. Copy the latest customizations
— After the testing environment has been restored to the point that
matches the production environment (SharePoint and production
customization version), you can start the server and copy the latest
solution packages (WSP) and any other customizations for deployment
purposes.
5. Upgrade the existing deployment
— In this phase, you perform the upgrade of the environment with your
latest build. Upgrade actions to perform depend on the project,
solution artifacts, and your development model. However, upgrade
solution commands are commonly performed from Windows PowerShell (as
done during the production environment upgrade). If new site structures
are required for the new customizations, you create the necessary
scripts to make these structures available for testing purposes, and
these scripts will be executed in this phase.
6. Start automated tests
— In this phase, start any automated test scripts that verify the
functional integrity of the code. Multiple different tools are
available for recording this kind of web test. If you use Visual
Studio, you can integrate the tests and results directly with the
source code projects.
NOTE Visual
Studio 2012 Team System provides project templates and test tools to
create, automate, and execute tests against your test environment.
7. Do manual tests
— All the previous steps can be automated to be performed every night,
or even multiple times each day. After automated testing has been
performed, you can continue performing manual tests. In larger
projects, dedicated test resources are responsible for reporting daily
build results and performing the manual actions required. If automated
builds are performed during the night, the latest version of the
package installed would be ready for the tester to continue testing the
next morning.
Automated builds can provide much more than just
integration testing of the customizations and code. You can use a
similar model before the initial version is available to re-create the
portal on a daily basis. For example, you could create Windows
PowerShell scripts that would create the initial hierarchy for the
future intranet. After developers update the web templates with the
latest changes, you would test the latest features on a daily basis.
However, the primary purpose for this kind of daily build environment
is not to create content because all content would be deleted or
re-created every day.
If you have requirements
in your projects to provide a long-term testing environment for your
customer, a target server for your daily builds would not be it. Good
practices for these kinds of requirements would be to have a separate
weekly release cycle for the environment, which can then be used for
functional testing by your customer.
