ACS OASIS Frequently Asked Questions

The ACS OASIS assessment is administered at the institutions enrolled in the program. While the ACS provides instructions to facilitate the assessment, individual programs are responsible for arranging the time and space within their local institutions to administer the assessment. It is anticipated that most enrolled programs will use a dedicated simulation lab space within their institution.

Programs enrolled in the ACS OASIS will receive assessment instruments for each of the eight technical skills stations. Each enrolled program is responsible for recruiting faculty members from their program to assess their residents upon the completion of surgical tasks on each OASIS simulation station.

The ACS OASIS is not a high-stakes assessment. The ACS OASIS was developed as a formative technical skills assessment to provide guidance on areas of strength and areas for improvement for surgery residents at the end of their PGY-2 year.

The ACS OASIS is designed to be administered to PGY-2 residents only once. Participating programs can determine if extra practice is warranted for specific residents based on the individual and program benchmark reports provided by the ACS. The ACS does not offer the opportunity to reassess residents of participating programs.

Yes, ACS OASIS is only available for programs that will be enrolling all their PGY-2 residents in the program.

During the pilot of the ACS OASIS program, participating site program staff responsible for building the models were surveyed to understand how easily they could acquire the materials and construct the models. Respondents generally reported that it was easy to obtain the materials and make the models. Staff were also asked approximately how long it took to make each station’s model. Four of the stations were reported to take roughly 15-20 minutes each to build, while three stations were reported to take about 35-45 minutes each to create. The central line station requests that programs utilize commercially available models and, therefore, does not require in-house development.

The ACS does not directly sell ready-made ACS OASIS simulation models or ready-to-build material kits. The models were designed to be created with materials that can be ordered online and/or obtained from local hobby, craft, and hardware stores.

Preparation time may vary among participating institutions due to differences in resource availability and staff familiarity with creating simulation models. We recommend allowing at least 2-3 weeks to purchase the necessary materials for creating the models. Once all required materials are ready, it typically takes one to two days to make all the models. A couple of weeks before the assessment administration window opens, programs will be given access to the assessment instruments for the eight stations. Programs should anticipate allowing time for participating faculty members to orient themselves to the assessment instruments.

The time structure for administering the assessment will depend on the number of residents being assessed and the number of faculty assessors available within a participating program. Each technical skills station is designed to take 20 minutes per resident, with the following structure:

1 Minute: The resident reads the station objectives and stem.
12 Minutes: The resident performs the surgical skills task.
1 Minute: The faculty assessor gives brief feedback to the resident.
6 Minutes: The faculty assessor completes the assessment instrument.

View an example schedule for the administration of ACS OASIS for a residency program with four PGY-2 residents and four faculty assessors.

De-identified resident assessment data is entered into the ACS REDCap database after the participating program has completed the administration of the assessment. This process may take 20-30 minutes per resident for checklist data entry. It is recommended to administer the assessment using paper copies of the assessment instruments, which can then be entered into the ACS REDCap database by a program staff member. Due to the de-identified nature of the data, the participating program must maintain a record of its unique codes mapped to its residents, as the reports received from the ACS will be labeled with the unique codes. The ACS does not collect or maintain any identifying information about the residents of participating programs.

Participating programs that do not submit their assessment data by the due date will be ineligible to receive program and individual resident benchmark reports from the ACS.

We defer this decision to each Program Director. The best approach will be having the resident assessed twice, once at the end of the PGY-2 and then again at the end of the research year to make sure that the resident has obtained all the required skill sets before completing the second year and that the resident maintains the skill sets after research years.