Telephone: (215) 716-7373


Home » Application & NetAnalysis » Baselining--Stress Testing--Performance Testing--Oh My--Part TWO


Baselining--Stress Testing--Performance Testing--Oh My--Part TWO

Bookmark and Share

Click Here to Listen to this topic as a "The ROOT Cause" podcast.  

 

Click Here to Subscribe through

 

Click Here to Subscribe through

 


This is the second of two articles discussing the topic of Test Environments and Testing Practices. The first one, "Baselining--Stress Testing--Performance Testing--Oh My--Part One--Environments," focused on proper testing environment design. This article is focused on what you do with them once they have been created--Network & Application Testing.

When one is performance testing, what is REALLY being tested? Is it the Application, the Network, the Desktop, the WAN? Having done a performance test of one, do you then know something about the others?  Not really.

What is the difference between Stress Testing, Load Testing and Baselining? Are they just different ways to say the same thing? No. 

What, if any, is the consequence of using these terms interchangeably?  After all, we all know what we mean don't we? It is common for various teams that are really trying to cooperate with each other to founder simply because they thought that they had agreed with each other--but discovered that each of them had a very different understanding of what is was to which they had all agreed. Semantics matter. This is not limited to IT by any means, yet, this article is focused on IT--one very specific part of IT, namely the processes used by IT to Test Performance.

TESTING DEFINITIONS

Baseline Testing: Focus is on how long a single transaction, or set of transactions, take for a normal user during normal conditions.

Only a single user is tested. Best if can be done SAFELY during production hours. After all, that is when users will experience any issues. A baseline that tells how the transaction functions under ideal conditions will only make any production environment seem too slow.

  • Test from each satellite location (if applicable). Use a Packet- Sniffer running locally to the test user. Something like Wireshark running on the host PC--if using live test users.
  • WireShark listening to a port mirror in a switchhub in front of the device running the script--if using a form of virtual user.
  • Capture the transaction while testing.
  • Save capture, test again.

BESTline Testing: (Our Term), describes a test that is performed to determine the best possible time an application can be reasonably expected to provide.

Attempts to eliminate all non-application issues such as the Network and WAN.

  • Use workstation most local to first tier servers.
  • If possible, relocate switch port for workstation onto the same ASIC.
  • If possible, will recreate the second and other tiers, as locally to the first tier as possible.

Application Profiling: Provides a packet level and protocol level description of what is taking place with an application--across a network wire.

  • How many TCP connections are used?
  • What is the nature of the communication at the packet level?
  • Many small connections or a few large connections?
  • What form of SMB is used?
  • What is the size of the files it “must” transport such as DLLs, EXEs, Java Applets, etc.
  • What are the size of required Cookies and Tokens?
  • HTTP usage For example, POST or GET? etc. URLs May also result in an INTERPATH APPLICATION FLOW DIAGRAM.

Application Profiling will be the topic of a dedicated article/podcast soon.

Performance Testing: Focus is on DEV, QA, or UAT.

  • Stress Testing: Attempts to see how much a system can handle before the system degrades below acceptable parameters or fails. Utilizies simulated users generated by a tool such as LoadRunner, or some other tool that generates virtual users.
  • Load Testing: Different than Stress Testing in that it attempts to induce Minimal Load. Steered towards a specific target based on a Capacity Planning goal.
  • Single Transactional Testing: Attempts to see exactly what is happening in a transaction from the packet level. May involve many monitoring locations along the application’s pathway. Requires in-depth packet analysis, but provides the best possible vision of how the application is performing the transaction(s) in question.

If one team is planning a Stress Test, but calling it a Baseline Test-- would it matter? Well, do they intend to try it in PROD? Ouch! You probably wouldn't want any kind of load in PROD. What if they really meant Baseline, but said Stress Test? While their actions would be safe, SOMEONE would jump up on a chair and say "stop!" That could damage team rapport and cause unnecessary delays.

Creating valid testing environments and then designing appropriate test plans are critical to creating stable architecture and applications. Many IT organizations of all sizes do not have what they need in this regard.

What about a real lab? Do you do any form of Device Certification on new load balancers, or WAN Optimizers, or Switches, or Routers, or Firewalls, or NIDS, or other Testing Tools themselves? Accurate testing, (accurate being the key word here) is far less common than it should be.

Of course, now you have another question. What about Capacity Planning?

 




Related Topics:


Back to main topic: Application & NetAnalysis
The Application's Interpath
Network & Application Performance Analysis Fundamentals
Building a Simple Open-Source Distributed Packet-Sniffer
Performance Tuning Is A Process -- Not A Tool
Excessive TCP Connections
Why Network Assessments Must Include Application Behavior
The 7 Most Common Mistakes Using Packet-Sniffers
Packet-Sniffer Filtering Concepts-01
Interpath Application Flow Diagrams-01
Baselining--Stress Testing--Performance Testing--Oh My--Part ONE
The Saturation Point
Multi-Tier Latency Concepts-01
Application Profiling Concepts - Part One

Subscribe to receive the latest news & specials on our products:

Quick Find
 
Use keywords to find the product you are looking for.
Advanced Search

Articles
New Articles
All Topics
 
 Application & NetAnalysis
 Case Studies .
 NetAnalysis Myth Series
 Podcasts
 Team Building
Articles RSS Feed
Information
Shipping & Returns
Privacy Notice
Conditions of Use
Newsletter
Contact Us
Catalog Feed

Copyright © 1999-2013 Barry Koplowitz