more-arw search

Q&A Forum

When would you NOT set up a test system/sandbox?

This is not an academic question:) If you are implementing a new accounting/ERP solution, which includes the need to make some important configuration settings, adding user defined fields and related workflows, would you try to develop/test/go live in only a production environment? Or, would you set up a sandbox/test system and test these changes rigorously before converting over to your production ("live") environment? What have you done in the past/what experiences did you have/lessons did you learn?


Title: CFO
Company: C-Suite Services
LinkedIn Profile
(CFO, C-Suite Services) |

In general...

If the system is heavily developed or configured for the company I would recommend a separate test system. If the system is a known quantity or has been in the market long enough, then most of the issues one may have can just be a result of implementation anxiety (which is normal). I.e. the providers usually know where the kink/problem is and easily corrected.

That being said, It also depends on how one perceives and know his/her pre-implementation plans are going. If one has a comprehensive (or a perception of it) plan and is going well (dotting the i's and crossing the t's) then i would recommend just going live. User Signoffs for every stage is important too.

I would recommend a comprehensive review of data migration (from the old system) and another round of review of the settings. I also recommend a Plan B (hang on to the old system) or C in case everything falls apart.

Topic Expert
Wayne Spivak
Title: President & CFO
LinkedIn Profile
(President & CFO, |

The vendor is incompetent.

Your never do major (and really shouldn't do) minor programming on a live system.

It is a recipe for disaster and contamination of data and programing should the scripts written fail.

Lyn Spens
Title: Consulting Manager
Company: Tyler Technologies
LinkedIn Profile
(Consulting Manager, Tyler Technologies) |

I can't imagine not having a test environment where you can heavily test the configuration before going live. Since I have been doing system implementations with clients using our proprietary software for the last 24 years, I think I have the experience to back up that statement.

However, some types of testing should be done only in the live environment. After final configuration decisions have been made and tested, they should be set up in the live environment. And then they should be tested there as well, by doing parallel testing. I have found that doing parallels in a test environment can turn into a nightmare if you are trying to make sure any data or setup tweaks are also done in the live system.

I also agree with Emerson that you get strong user signoffs for every phase of the implementation. They should never be a pro-forma signoff. The user needs to be completely comfortable that their system operates as intended. I have actually refused to let a client sign off on occasion if I felt that they really did not understand what they were signing - even if it delayed the go-live.

Dr. Rod Sparks
Title: Consultant
Company: IMPACT Solutions
(Consultant, IMPACT Solutions) |

Doing a major ERP conversion without adequate planning/testing is an invitation for disaster. I worked with a major hospital system that apparently tried to shortcut testing, went live on their new system, and then discovered they couldn't write vendor checks ... for several weeks!

Mark Halpren
Title: Principal
Company: FirstCFO Inc.
(Principal, FirstCFO Inc.) |

I have implemented a cloud based software solution into a live system where any transactions will be clearly marked as originating from the software, volume of transactions will be very low by limiting scope initially and transactions can be reversed or deleted if errors occur. I agreed to do this because we would lose weeks of programming time and incur additional costs by running on our test environment. It really becomes a cost / risk / benefit analysis.

Vincent Gismondi
Title: President
Company: Enterprise Resource Consulting
(President, Enterprise Resource Consulting) |

Well said everyone. A test environment doesn’t take time – it saves time. Almost every programming change, no matter how small, goes through more than one iteration. If the data becomes corrupted along the way, programmers will inevitably be chasing after bugs that don’t exist. A test environment also allows users to try things they would never risk on live data.

Geoff Coleman
Title: VP Product Strategy
Company: Transverse LLC
(VP Product Strategy, Transverse LLC) |

From the vendor side of the fence I would always insist on a test system where everything is vetted out and where all future changes can also be tested when required.

Ern Miller
Title: Co-CEO
Company: Miller Small Business Solutions
(Co-CEO, Miller Small Business Solutions) |

To answer the question, I would NEVER - EVER - NEVER agree to install anything without testing, except on my own system.

That is because I am willing to take full responsibility for anything that goes wrong. I can control everything, so, I can be wary of what I do to cause problems.

With anyone else, I am NOT doing anything without testing. Too many variables. Plus, you have a person who will focus responsibility on me if anything goes wrong, even if it was caused by the Trojan in the latest cat video they watched.

When things go wrong, depending on the severity of the problem(s), someone's job could be on the line. When jobs are at risk, everyone clams up and has amnesia about what they did, covering up any evidence while stonewalling.

Though computers can hold a lot of records, those records can disappear in an instant. Those records that would have protected you, and pointed the responsibility in the right direction.

This is how things should work:
1. All responsible parties involved should agree to sign off at their stage of the process.
a. Explain that signing off does not mean they are accepting that the previous stage is working 100%.
b. They are signing off that everything expected in their stage of implementation is complete and working. (100%. If part of their responsibility is done and must be progressed while another part is not ready, before progressing anything, split up the two stages and follow them separately.)
c. DO NOT move ahead on any stage until the previous stage is signed off. Refuse to allow moving ahead, even if your job is on the line, because if you allow progression of a project and it fails, your job IS on the line.

2. If someone forces progression before signoffs, then require them to take responsibility, along with recorded evidence that you are moving the project forward under protest. If they question you about the paper trail, explain to them that by having a record, you can prepare to respond quickly should anything go wrong. By saying that, you will either get them to wait, or will alleviate fears that you are making a paper trail to cover yourself and point responsibility on them.

Now, here's how things should develop:
1. Determine specs for the project
2. Design project to specs
3. Build project
4. Test project
a. While testing, begin training your SMEs
5. Implement project
6. Monitor project
7. Retire in the Bahamas or Hawaii

The Test cycle is this:
1. Run Test
2. Evaluate results
3. If test passes, begin implementation. If test fails, redesign and build and do Step 1 again.

Repeat until all tests are passed.

Implementation goes easy and quickly in a ratio equivalent to the level of adherence to testing...that is, the more thorough the testing, the easier the implementation.

Now, about ready-made products. Would I install Microsoft Office without testing?

Sure. But not because I ignored the previous rules laid out. Rather, it is because Microsoft did the testing for me. They have invested a lot of money to make sure their product is stable, and functions exactly as expected.

Packaged products do not eliminate the need for testing. Instead you are paying the manufacturer for already doing the testing.

Dan Johnson
Title: BDO
Company: GraVoc
(BDO, GraVoc) |

What Wayne said. I assume an "Eecipe" is his code term for "Extreme Recipe" for disaster and contamination. Is there another industry that submits product to market and/or end user without testing? That would be madness right? Why is software different?

Mark Matheny
Title: VP - FInancial Planning and Analysis
Company: Novolex (formerly Hilex Poly)
(VP - FInancial Planning and Analysis, Novolex (formerly Hilex Poly)) |

Test, test, and test again.

Topic Expert
Len Green
Title: Performance Improvement Consultant and E..
Company: Haygarth Consulting LLC
LinkedIn Profile
(Performance Improvement Consultant and ERP Strategist, Haygarth Consulting LLC) |

Thanks all, I appreciate your comments (with plenty of implicit warnings about risks!!), which makes me wonder why people take these risks time and time again.

End users may assume (and they may be trusting or naive) that their implementation partner will LEAD them with sound practices and tools to succeed. If that trust is misplaced, if the end users don't understand what they really need to do, then a failed implementation is highly likely.

Topic Expert
Barrett Peterson
Title: Senior Manager, Actg Stnds & Analysis
Company: TTX
(Senior Manager, Actg Stnds & Analysis, TTX) |

A development sandbox, always. We even have an intermiediate "quality testing" environment between "dev" and production ["prod"[.

Topic Expert
Bob Scarborough
Title: CEO
Company: Tensoft, Inc.
(CEO, Tensoft, Inc.) |

We work with many technology companies - often from early revenue stages through significant growth. One of our required skills is advisory capability related to scaling customer organizations and business processes.

We use the following criteria to decide when user acceptance testing is required by their company:
1) When the system use is mission critical - and any substantial downtime is unacceptable.
2) When the transaction volume is high enough to prevent real time trouble shooting.
3) When the company has outgrown the power user model of financial system / ERP support
4) When the business process complexity has moved from the people to the system

Sometimes the challenge is to explain why something was acceptable when the company was smaller - and is no longer possible or realistic today.

Bob Scarborough

Products and Companies

Get Free Membership

By signing up, you will receive emails from Proformative regarding Proformative programs, events, community news and activity. You can withdraw your consent at any time. Contact Us.

Business Exchange

Browse the Business Exchange to find information, resources and peer reviews to help you select the right solution for your business.

Learn more

Contribute to Community

If you’re interested in learning more about contributing to your Proformative community, we have many ways for you to get involved. Please email [email protected] to learn more about becoming a speaker or contributing to the blogs/Q&A Forum.