CHAPTER III: Beta Testing

In this chapter:
-General beta testing regulations
-Additional regulations
-Becoming a beta tester
Sample beta testing reports


In the early years of the Emperor’s Hammer, all submitted battles were immediately released by adding them to EH Newsletter. Mission creation was very poor and often the only way to do it was to use a hex editing tool. However, with the evolution of the EH and creation of the Mission Compendium, Tactical Office had to develop better ways to check new submissions and eventually correct them. That’s how the beta testing process was initiated.

3.1 General beta testing regulations:

When you e-mail your battle to the Tactical Officer, it will be initially reviewed to see that the basic standards have been met. If the battle is missing a plotline or an .LFD file or .LST file or has some other problem with it, the Tactical Officer will notify you of the problems and require you to resubmit the battle again once the problems have been fixed. Make sure to follow the submission procedures or your submission will be simply rejected.

If the first check passes, the battle will be mailed out to the Tactical Head Coordinator and the appropriate Tactical Coordinator (i.e. TCC:TIE for a TIE battle). The battle will be placed in the Beta Testing Queue and will await the next beta testing assignments. Battles are reviewed in the order they are received.

When a battle reaches the end of the testing queue, the Tactical Coordinator will assign Tactical Surveyors to test the new battle. Beta Testing assignments vary in length, but could go anywhere from one to three weeks.

Once the play testing has been done, all beta test reports goes to the Tactical coordinator (Tactical Head Coordinator as well for archives purpose). The Tactical Coordinator puts the reports together and approves or rejects the battle based on the volume of bugs present. Three beta reports are needed (one for each difficulty level, from different Tactical Surveyors) before the battle can be rejected or approved. If the battle is approved, the Tactical Coordinator hands them to a Tactician to make corrections on any bugs the Tactical Surveyors find.

The Tactical Coordinator will work with the Tacticians to correct any bugs in the battle. Once the battle has been corrected, the Tactical Coordinator play-tests it one last time before sending it to the Tactical Head Coordinator and Tactical Officer for final review.

If there are severe bugs and errors that make the battle unplayable without a few simple fixes (i.e. not issuing any waypoints so all the missions would crash), the battle will be rejected. The author will be notified and will receive a copy of the Beta Test Reports outlining what needs to be fixed. Authors can ask to correct their battle themselves rather than having it done by a Tactician, however the Tactical Coordinator has the final say on correction if some bugs are still present after the author’s corrections.

Finally, the Tactical Head Coordinator and/or Tactical Officer will then play the battle once through to ensure the bugs have been corrected and that the battle plays properly. Finally, the completed battle will become official, be added to the Mission Compendium, and the author will receive the appropriate award(s) for creating the battle by the Tactical Officer and the Tactical Staff will receive credits for playtesting (battle added to their combat profile) and Medals of Tactics for correction from the Tactical Head Coordinator. Merit medals may also be awarded by the Tactical Head Coordinator.

3.2 Additional regulations:

Additional regulations concern Tactical Office staff. They serve the rules and guidelines for all testers and engineers.

3.2.1 Tactical Surveyor

Every week or so, you will receive a battle to play from your Tactical Coordinator. You will be assigned to playtest it as a specific difficulty level. You are *required* to play that battle for the week. Email your beta reports to your Tactical Coordinator (cc as well the Tactical Head Coordinator). You may have from one to three weeks to playtest the battle or set of free missions, depending on your Tactical Coordinator and what you may have defined with him/her.

If for any reason you cannot test it for the deadline or need to go on leave for any reason, you *must* notify the Tactical Coordinator or you will be marked with an unexcused miss. 3 unexcused misses and you may be removed off the team.

If you complete your assignment before the deadline, submit your Beta Report to the Tactical Coordinator (CCed to the Tactical Head Coordinator). Then you can either:
· Ask for a new assignment as soon as possible.
· Ask for a rest until the end of your current deadline.
· For example, if you have been given 14 days to playtest a battle, and if you have playtested it and written your beta report in 5 days, you can either ask for a new assignment, or ask for a rest for the 9 remaining days. Remember your awards as Tactical Surveyor will depend on your amount of work.
· Tactical Surveyors are not required to pass the appropriate mission creation course, but it is strongly recommended, as it gives a better knowledge of the way a battle works, and will improve the quality and precision of your beta reports.

Here are some additional guidelines:

· Play on your assigned difficulty level. If the mission is too difficult to play on your difficulty level, turn on "cheat mode" and state in your report you had to use the in-game cheats to beat the level. If it's still too difficult, you may move down to the next difficulty level and state that in your report! If the mission is too difficult to play on EASY mode with "cheat mode", then report it as a bug and skip to the next mission.

· In TIE, BoP, or XWA, you must **lose** all missions (Use "Q-Space" at the very beginning of the mission, don't eject) at least once to view the "hints" briefing. In TIE, make every effort you can to complete the Primary AND Secondary goals so you can view the "success" briefings for both. If you can't complete the Secondary goals, state it and your reason why in your report.

· If you come across a problem such as a corrupt .ZIP file or a mislabeled battle showing the wrong game platform which makes the battle completely unplayable, notify the Tactical Coordinator IMMEDATELY with a copy of your report in progress. If reported soon enough, a fixed version (if it can be fixed) can be distributed to all members. Be sure to write "*URGENT* or *BATTLE UNPLAYABLE*" in the subject header so the Tactical Coordinator sees it ASAP!

· If a particular mission is causing problems and you can't complete it, record the bugs down in your report and skip the mission. In TIE Fighter, use a pilot editor (TFW or BSF Generator) to skip the mission. In XvT/BoP, simply play the next mission in line. In XWA, use one of your in-game skips or a pilot editor. If you need a pilot editor, check the Utilities section of this manual.

· If the battle you play is approved on the Battle Center, you will earn credit for completing it. But you may not use your Beta Testing pilot file to turn in a high score! You must play the official distribution version of the battle to achieve a high score. ***BE CAREFUL OF THIS!*** You don't want to be accused of cheating by your superior because you turned in your beta testing pilot file by accident!

· Tactical Surveyors are not required to turn in pilot files (anymore) as proof of completion, so there's no reason to keep your old Beta Testing pilot files on your hard drive.

· MAKE SURE THE GAME PLATFORM YOU'RE USING IS UNMODIFIED! If you installed any custom patches or add-ons, you must remove them! LucasArts-released patches and other patches on the Updates section are an exception to this rule. Some battles may ask for you to use a custom patch to play it. Make sure you uninstall these patches or completely reinstall your original game (that's the best way) before playtesting other battles.

· Do NOT play battles that have been denied! If the Tactical Officer or Tactical Head Coordinator sends a notice saying a battle has been prematurely denied for a reason, you are excused from having to play it and may move on to play other battles.

· Grammatical errors and misspellings should be noted on reports! The same goes for the text-running-off-screen error. Be specific to where these are. (i.e. 2nd map statement "assalt" misspelled; 3rd briefing officer question, page 2, "fihgters" misspelled; secret order failed debriefing, line 5 runs off screen)

· Check for appropriate content. This includes checking for anything that might violate EH Bylaws or EH AoWs OR if you feel in some way that the battle is not Emperor's Hammer related. Battles should be suitable for pilots of ALL ages. If you feel something is offensive or inappropriate, state it in your report as to why. The Tactical Officer will make a final judgment call on the appropriateness after viewing the reports. [I've been pretty lenient in the past, but that led to complaints later. I'd like to avoid "controversial" subject matter.] If you see swearing, even if they bleep out a letter or two, make a note of it anyway. Racial and Ethnic slurs in any form outside of the Star Wars universe (bleeped out or otherwise) are inappropriate.

· Every mission you review MUST list a bug or else the review doesn't count. If you're 100% sure it's bug free, you must put in additional review comments to say *something* about the mission (and particularly, something specific you liked/disliked about it). This is how the Tactical Coordinator can tell you actually played the battle.

· If a battle forces you to wait excessively for more than five minutes (x4-time in TIE or real-time in XvT/BoP/XWA), report it as a bug. You may choose to skip the mission if excessive waiting is involved since waiting there for that long is a waste of time.

· With Free Mission Sets, you must play ALL the missions in the Free Mission set to get credit. Please submit reports for a set TOGETHER. Do not just do one mission at a time and e-mail the Tactical Coordinator a separate report for each. Make a full report for each Free Mission and put them back-to-back.

· If you are accidentally assigned to test your own battle that you made, please notify the Tactical Coordinator for reassignment.

Reports go to the Tactical Coordinator, CCed to the Tactical Head Coordinator.

**PLEASE NOTE THAT THERE MAY BE NO TACTICAL COORDINATOR OR HEAD COORDINATOR, IN THIS CASE YOU E-MAIL THE TACTICAL OFFICER**


3.2.2 Tacticians

Every week or so, you will receive a battles to correct from your Tactical Coordinator along with a set of reports from the Tactical Surveyors. You may have from one to three weeks to correct the battle or set of free missions, depending on your Tactical Coordinator and what you may have defined with him/her.

If for any reason you cannot fix the battle for the deadline or need to go on leave for any reason, you *must* notify the Tactical Coordinator or you will be marked with an unexcused miss. 3 unexcused misses and you may be removed off the team.

If you complete your assignment before the deadline, submit your corrected version to the Tactical Coordinator. Then you can either:

· Ask for a new assignment as soon as possible.
· Ask for a rest until the end of your current deadline.
· For example, if you have been given 14 days to fix a battle, and if you have fixed it in 5 days, you can either ask for a new assignment, or ask for a rest for the 9 remaining days. Remember your awards as Tactician will depend on your amount of work.
· Tacticians are required to pass the appropriate mission creation course from IWATS.

Here are some additional guidelines:

· Document all changes you make to the battle at the bottom of the battle's update.txt file with a date stamp and your name:

-----------------------------------------------------------------
- CRS Runaway not showing up bug
- Added difficulty levels to starfighter groups
- Blank questions removed
by CPT Wlodek 07.22.2000

· Add difficulty levels or reinforcement Flight Groups if necessary if complaints of the battle being too difficult are present. Be VERY cautious that you don't assign a mission-critical or plot-critical ship a difficulty level that makes the battle unplayable or disrupts the plotline.

· Playing with unlimited ammo and invulnerability is permitted to speed up time with testing corrections.

· Common sense is required for this job. Use it! Don't blindly make every bug correction reported by testers that are unnecessary. However, take in any suggestions made to make the battle better. A skilled corrector can take a 0 or 1 rated battle and turn it into a 4 or 5 rated battle.

· Bad quality radio messages in .WAV format do not have to be rerecorded by Tacticians.

The following are commonly reported bugs that need to be corrected:
· Spelling and grammar mistakes in briefings, descriptions, radio messages
· Secret Order Briefing Officer present without any questions for the SO (fix by removing the SO, not by adding questions)
· Blank spaces in briefings
· Capital Ships with starfighter orders (while annoying, it's not necessarily a bug)
· Impossible difficulty on Easy mode, lack of difficulty settings
· Long wait times
· Unbeatable primary mission goals (unbeatable secondary and bonus goals are acceptable, but correct them if time anyway)


3.2.3 Tactical Coordinators

There is a Tactical Coordinator for each game platform.

Every week or so, you will receive battles to playtest/correct from the Tactical Head Coordinator or the Tactical Officer.

The Tactical Coordinators have several responsibilities:

· Evenly distribute Beta Testing battles to the Tactical Surveyors and assign difficulty levels.
· Collect all reports from Tactical Surveyors and document which battles the Tactical Surveyors playtested.
· Assign playtested battles to a Tactician to see that it gets fixed, once three beta reports have been received by the Tactical Surveyors (once per difficulty level, each from a different Tactical Surveyor); forward the compiled beta reports to the Tactician.
· All assignments must be communicated to the Tactical Head Coordinator for the general management of the Tactical Staff.
· When you receive a corrected battle playtest it one last time at your preferred difficulty level. If more corrections are required, send it back down to the Tactician, otherwise, the final battle goes to the Tactical Head Coordinator / Tactical Officer.

Make sure to keep a roster of all Tactical Surveyors and Tacticians for your game platform. An updated roster should be sent to the Tactical Head Coordinator once a week, along with updated status on the battles in your office (i.e. still in the Beta Queue, under playtesting, or under correction).

Authors may have asked to correct themselves their own battles. In this case, rather than sending the battle with beta reports to a Tactician, send them to the Author. If the author's corrections are not enough, you may then correct yourself what is still wrong or assign a Tactician.

Assignments to Tactical Surveyors and Tacticians can be from one to three weeks, depending on you and what you may have defined with them (eventually case by case). You are given flexibility in the way you run assignments, just make sure the work is done...

If an assignment is completed before the deadline, the Tactical Surveyor or Tactician can either:
· Ask for a new assignment as soon as possible.
· Ask for a rest until the end of his/her current deadline.
· For example, if he/she has been given 14 days to playtest a battle, and if he/she has playtested it in 5 days, he/she can either ask for a new assignment, or ask for a rest for the 9 remaining days.
· Remember you do not have the authority to award any medal or any FCHG points. Only the Tactical Head Coordinator or the Tactical Officer are able to reward the members of the Tactical Staff, based on a merit system defined for the whole Tactical Staff.

Tactical Coordinators are required to pass the appropriate mission creation course from IWATS.

3.2.4 Tactical Head Coordinator

There is only one Tactical Head Coordinator. You will be the only member of the Tactical Staff (excluding your superiors) that will have the authority to award medals.

The Tactical Head Coordinator has several responsibilities:

· Playtests the final version of battles prior to official release.
· Awards Tactical Surveyors the appropriate medals for that beta testing.
· Fills out BSFs for Tactical Surveyors and Tacticians (Tactical Staff BSFs will always have a score of 0 with "Beta Testing" as its comment) when the Tactical Officer approves a new battle.
· Maintains a roster of all Tactical Staff Members and what platforms they are capable of testing along with collecting rosters from the Tactical Coordinators, sends an updated roster to the Tactical Officer once a week.

Tacticians are to be awarded a Medal of Tactics - Blue Hammer per battle or set of five missions corrected. Merit awards are to be awarded following a point system, which must be approved by the Tactical Officer. An example is as follows:
· +1 point for every mission flown
· +1 or 2 points for every mission corrected, depending on corrections
· -5 points for every assignment NOT completed

The Tactical Head Coordinator is required to pass all available mission creation courses from IWATS.


3.3 Becoming a beta tester:

When new openings are available for Beta Testing positions, the Tactical Officer will announce open applications on TIE Corps News and how to apply. At that point, it typically "first come, first serve." Those who have some skills in mission creation and passed the appropriate mission creation courses have a significantly better chance of becoming testers, even if they apply for a Tactical Surveyor position. All Tacticians are required to pass the appropriate mission creation course from IWATS, Tactical Surveyors are strongly recommended to do so.

Beta Testing is a huge commitment and requires a good amount of work per week. The rewards of beta testing are great. Merit medals are awarded depending on how many battles pilots has tested or corrected, FCHG points are awarded for playtesting once the battles are officially released, and Medals of Tactics are handled for bug fixes.

3.4 Sample reports:

Sample reports are the best examples of beta testing reports, placed here to show you how “the best there are” look like. Basically, with tactical database 2.0 the testers are simply required to fill in the blanks. But should you have to e-mail the report to the Tactical Officer or a TCC, it would look like this.

3.4.1 Sample beta testing report

--------------------------------------------------
------- Emperor's Hammer Beta Testing Form -------
---------------- X-Wing Alliance -----------------
--------------------------------------------------

Battle Name: TIE Fighter: Demonstration
Battle Type (Subgroup): TIE Corps (TC)
Assigned Difficulty Level: hard
Tester's ID Line: CMDR-TCS/COL Abel Malik/Praetorian/DGN Lichtor V
Tester's E-mail: ezwill@famvid.com

--------------------------------------------------

[X] Readme.txt
[ ] Battle medal name (n/a this being a FXWA mission)
[X] Plotline/Storyline
[X] .lst file(s)
[ ] EH custom ship patches needed
[X] Custom EH Crest present
[X] Extra installation instructions needed (info on where to intall the 8P COMPLEX TIE FIGHTER DEMO.SKM might be nice);p.
[ ] Extra installation instructions included

Plotline/Storyline problems: no problems
Readme.txt problems: no problems
Other text files' problems: n/a

**************************************************

Mission #1- TIE Fighter: Demonstration

[X] Opening Text Present
[X] Animated Map Briefing Present
[X] Commander Briefing Present
[X] De-Briefing Present
[X] Hints Present
[X] Text Radio Messages Present
[ ] Custom Voice Add-on Messages Present
[X] Appropriate Content

Animated Briefing Errors: no errors present.

In-flight Text Messages Errors: no errors present.

Gameplay Errors: I didn't encounter any errors while testing the mission.

Additional Review Comments: the goal is the elimination of all disabled rebel craft at the repair yard near Remitik (rebel craft consists of y-wings & Lambda class SHU) the repair yard is made of CN/A storage containers. Its a simple matter to destroy the repair yard and the diabled craft; however, after accomplishing that goal many waves on incoming rebels evacuating Hoth hyper into the area (these are mostly Bulk Freighters, SHU's, and Correlian Corvettes escorted by y-wing, x-wing or a-wing starfighters). Its not as easy to destroy the incoming waves in an unshielded TIE fighter, but accomplishing the actual mission even on hard difficulty is relatively easy.

bonus points on hard difficulty:
3175 (for destroying all entering rebel craft)
3175 (for destroying all of A-wing Rogue W)
3175 (for destroying all of A-wing Rogue M)

player craft: TIE Fighter

**************************************************

Rating (1-5): 4
Additional Overall Notes/Review: This is a nice bug free mission with several opportunities to rack up bonus points for possible high scores.

CMDR-TCS/COL Abel Malik/Praetorian/DGN Lichtor V


3.4.2 Sample correction report

Correction reports aren’t held to very strict standards. A simple list of all corrections and in which missions the corrections were made would do very nicely.


Back to table of contents