Package Bidding System Arizona Test Results As part of the development of the FCC Package Bidding System, a series of functional tests were performed externally to FCC to more thoroughly examine the accuracy and usability of the system. The initial round of Package Bidding System testing was conducted in the economics labs of the University of Arizona from August 16 through August 18. Tests were conducted between the hours of 3:00PM and 7:00PM EST to accommodate the time zone difference in Arizona. Tests were conducted under the on-site guidance of Dr. David Porter, Senior Research Scientist at the University of Arizona Economic Science Laboratory, and Karen Wrege, Package Bidding System development manager from FCC. Operations for the test auction were overseen at the FCC facility in Washington, DC by technical staff from Computech and their subcontractors. A team of ten graduate students studying economics at the university was used to submit bids during the test. Before bidding started, test coordinators trained the bidders on the rules of the auction and the details of how the combinatorial process would work. Each bidder was given a set of parameters to guide their bidding behavior, including budget and license interests. Bidders were free to choose their own strategy, with their assigned parameters, and were given incentives to behave as realistically as possible. The test auction opened up with a 10% bid increment factor and a 1/2 activity requirement. An attempt was made mid-auction to alter these factors to 7% and 2/3, respectively, but this effort failed due to a defect in the system. The auction continued through to completion with the original factors in place. Auction Summary ? Simulating Auction 31, there were 12 license available in the auction, divided into 6 10MHz and 6 20MHz licenses. ? All bidding rules designed for Auction 31, as defined in the applicable Public Notice, were applied to the test. ? There were 20 test bidders actively participating in the auction (several testers participated as more than one bidder simultaneously) ? The auction closed naturally after round 26 ? There were 6,860 bids placed on separate licenses or packages. ? The winning bid was the global package, bid in round 19 for $1,861,648 and renewed repeatedly until the auction closed. ? 17 bidders used all of their waivers and had their eligibility reduced to 0, 2 bidders used some of their waivers, and 1 bidder used none of their waivers. ? All bidders started the auction with 252,000,000 bid units of eligibility. All bidders but the eventual winner were reduced, either automatically or by choice, to 0 eligibility by the end of the auction. Test Findings After each day of the testing the Arizona test coordinators compiled a list of test findings and delivered them to the Package Bidding System development team. These findings included questions about the rules, observations of unexpected system behaviors, observations on the utility of the auction design, and challenges of the systems calculations. Each day the development team conducted a triage session, during which each observation was discussed and a preliminary response formed and communicated via conference call to the Arizona test coordinators. The unfiltered test observations resulting from the test are listed below. ? Bidder 4 could create more than 12 packages ? Many bidders had a problem with duplicate packages ? Sometimes Bidder 8 could create packages with “+” signs in the names ? At the original start of the auction subjects could not bid on packages with grater than 2 licenses ? For packages, the minimum bid was not the sum of the individual item minimum bids when the auction originally started ? The clocks at the bottom of the screen were not the same for all bidders ? Sometimes the process of submitting and confirming took a long time. ? Eligibility can be increased ? Incorrect loss of waivers in certain cases ? Renewals of non-provisionally wining bids appears to generate activity ? In round 7 and bidder outbid another bidder’s round 6 bid on a license, but didn’t win. Why? ? Bidder 26’s eligibility in round 2 should be 885,500,000 but shows as 745,000,000. ? Renewals seem to count as new bids ? Files don’t note which bids are renewals and which bids are new ? Problems opening files using Netscape; IE works fine ? Results page indicates history for rounds 7-11 when in round 2 ? Bidder 9 tried to login but got a “Submit Query” display ? JavaScript errors near the end of the auction ? Results page failed to show all winners on one bidders screen ? Bidders can reduce their eligibility to 0 ? The number of remaining bidders on the results page should not show people with 0 eligibility ? The drop down boxes are different sizes are it looks bad ? It would be nice to see the current winning bid on my bid page ? It would be nice if the results would automatically default to map of your own bids ? When results are running, the refresh page shows, then takes you to the Home page. It would make more sense if it took you to the refreshed Results page ? When I enter, but don’t yet submit bids, then go to packages and come back, my entered bid have been cleared ? On the results page it would be great if the name of the region appeared next to the grid so I can tell what bid is for what ? When I have submitted bids up to my maximum eligibility, I am asked to confirm the bid. Then when I go to add another bid, which will go over my max eligibility, I get a message saying I exceeded my eligibility but I can’t see why.