RTN-064: JOR review recommendations

  • William O'Mullane

Latest Revision: 2023-07-26

JOR recommendations

Review recommendations extracted from Jira.

Download jor.csv here.

2023-07-28 00:02

Issue key

Rec#

Summary

Report date

Due Date

Implementation Status

Description

Response

Implementation Status Description

PREOPS-3542

M-III.6-25

Hold a risk register scrub/workshop

JOR_2023_02

2023-09-01

In Progress

Hold a dedicated risk register scrub/workshop with subsystem leads and risk owners as well experts in risk management, in addition to the upcoming cost scrub to ensure focus on risk and complete the risk register. This activity should engage all levels of management that will be involved in operations [by September 2023].

This has been completed in May 2023. The scrub is done and the RDO is updating budgets and Risks.

A pre-scrub risk workshop was held in April 2023, and served to homogenize and finalize the cost and schedule impact analysis of the risk register, as well as surface some new candidate risks. This workshop was only attended by the Risk Board, with no external experts in risk management.

During the scrub, we identified a need to capture and analyze opportunities, as well as threats, that could then be supported financially via requests beyond target.

PREOPS-3541

M-III.6-24

Develop, document, and socialize detailed decision-making processes

JOR_2023_02

2024-06-01

In Progress

Develop, document, and socialize the detailed decision-making processes for both short-term tactical choices and longer-term strategic operations choices that impact scientific results. This should include the specific lines of communication and decision-making authority within the operations team as well as how feedback is solicited and incorporated from science collaborations and other stakeholders [by one year before operations start].

We will do this. A start has been made in rtn-005.

PREOPS-3540

M-III.6-23

Change management plan and resourcing

JOR_2023_02

2024-03-01

In Progress

Ensure that sufficient resourcing is allocated during the pre-operations phase to allow all future operations staff to become engaged, regardless of their ongoing construction activities [by March 2024]. This includes but is not limited to: • Develop a plan to transition the existing internal project Science Team to operations, including detailing the processes for this group within the operations plan. • Continue work to solidify the processes that allow delegating responsibility to lower-level managers and enabling them to make appropriate decisions. • Deliver the change management plan that is understood and accepted by all staff with high priority.

We will do the first two bullets. It is not clear we need a change management plan so much as a transition activities that includes regular communication down to the individual team member, clarity and sign off on individual roles and responsibilities, etc.

PREOPS-3539

M-III.6-22

Best hiring practices for all Rubin hiring committees

JOR_2023_02

2024-03-01

In Progress

Complete and implement a set of best hiring practices for all Rubin hiring committees, regardless of parent organization for the hire, that includes within it best practices to ensure diversity, equity, and inclusion [by March 2024].

We will do this.

PREOPS-3538

M-III.6-21

Formalize advisory committee processes

JOR_2023_02

2024-03-01

In Progress

Formalize the make-up and meeting cadence of the advisory committees, particularly for the case of the science collaborations, to ensure the science collaborations are appropriately represented and that the process for acknowledging and responding to their advice is clear. Additionally, create a single unified organization chart that shows all advisory committees and the communication lines to management [by March 2024].

We will do this. Most of it already is documented in RDO-018.

PREOPS-3537

M-III.6-20

Develop salary equity process for AURA & SLAC

JOR_2023_02

2024-03-01

Not Started

Develop a process to ensure AURA and SLAC review salary and benefits at least annually to ensure similar compensation for similar positions and experience across the organizations [by March 2024].

This will depend on AURA and SLAC HR.

PREOPS-3536

M-III.6-19

Policy to allow employees to work from any state

JOR_2023_02

2024-02-01

In Progress

Make a policy change at AURA allowing for the hiring of employees to work in any state, without consideration for AURA’s existing presence in that state [by the next review].

Rubin will work with NOIRLab leadership to enable this change with AURA HQ. Rubin has already worked with NOIRLab to put in a formal request to add Oregon, NY, and North Carolina. This request is in AURAs hands.

PREOPS-3535

ESH-III.5-18

Lessons learned folded into ops

JOR_2023_02

2023-06-01

Implemented

Ensure that the lessons learned and recommendations from construction are folded into operations. • Complete the Winter Operations Plan and Response to Weather Emergency Situations plan well in advance of the 2023 winter season to allow time for equipment procurement, dry run exercises, testing of equipment [by June 2023]. • Develop a Fatigue Mitigation Plan. Suggest including fatigue observation checklists and assessments [by June 2023]. • Perform an annually quantitative measure of the impact of implementing recommendations from the external traffic safety company evaluation. [Complete the first one prior to the next operations review.]

Winter plan 2023 on place Fatigue Mitigation program on progress Safety Road Enginnerering Study (done) Corrective Measures in progress

Winter plan 2023 on place

Fatigue Mitigation program on progress

Safety Road Enginnerering Study (done) Corrective Measures in progress

PREOPS-3534

ESH-III.5-17

Document lessons learned

JOR_2023_02

2023-06-01

Implemented

Document the lessons learned recovering from a major setback such as resuming work after 2 months of shutdown following the incident that took place at SLAC in December 2022 and share with the Rubin team [by June 2023].

Safety Lesson Learned has been distributed

Lesson learned about SLAC december 2022 incident has been properly distributed . On parallel a reinforce of LOTO and electrical hazards has been carried out

PREOPS-3533

ESH-III.5-16

Plan to execute recommendations

JOR_2023_02

2023-09-01

Not Started

Create a plan for executing the recommendations from the ES&H review of the Rubin Observatory which took place on 10/6/22. Focus on electrical safety, confined spaces, ladder safety/working at heights and earthquake bracing [by September 2023].

Reinforce of :

Electrical Safety ; LOTO training update

Confined Sapce : Develop of a Confine Space Training

Working on height and Fall protecction : review of current traning and add of ladder safety

Reinforce of :

Electrical Safety ; LOTO training update

Confined Sapce : Develop of a Confine Space Training

Working on height and Fall protecction : review of current traning and add of ladder safety

PREOPS-3532

ESH-III.5-15

Implementation of Road Safety Study

JOR_2023_02

2023-07-01

Not Started

Deliver a timeline for implementing the recommendations from the Road Safety Study (Estudio de Seguridad Vial by Ambitrans Ingenieros Consultadores, SpA) report [by July 2023].

Minimun Commuting Time Test , March 2023 , done ,

Submit new Minimum Timing Table to AURA-O and RSLT (done)

GPS monitoring Submitted to AURA-O for approval (May 2023)

Distribution of new Minumum time comutting , June 2023

Minimun Commuting Time Test , March 2023 , done ,

Submit new Minimum Timing Table to AURA-O and RSLT (done)

GPS monitoring Submitted to AURA-O for approval (May 2023)

Distribution of new Minumum time comutting , June 2023

PREOPS-3531

CS-III.4-14

Training on new tools

JOR_2023_02

2024-01-31

In Progress

Provide sufficient training on new tools and budget and the budget planning process for ADs and team leads [prior to the launch of next annual planning cycle].

We will train the ADs on all the tools necessary to do their part of planning and tracking.

PREOPS-3530

CS-III.4-13

Consolidated report Plan vs Actual

JOR_2023_02

2023-08-31

In Progress

Develop a consolidated report showing plan versus actuals by WBS by month for the Rubin Operations team and agencies [by summer 2023].

We will make this report. The NSF side is essentially already done in collaboration with Program Operations and NOIRLab Management Services.

PREOPS-3529

SP-III.3-12

Track RSP usage

JOR_2023_02

2024-06-28

Not Started

Continue to find ways to track how broadly the Rubin data and RSP are being used by the community. In particular, assess the impact of the LSST survey data reaching historically underrepresented groups [before Data Preview 1].

The RPF Community Science team will investigate ways to track RSP usage

Added Tina Adair as a watcher. Within CST she’s leading development of our “annual reports” which might (if it’s possible) include tracking this kind of usage.

PREOPS-3528

SP-III.3-11

Visibility of change procedures for the survey cadence, scheduling and strategy

JOR_2023_02

2023-09-01

Not Started

Increase the visibility of the change procedures of the survey cadence, scheduling, and strategy to inform the community and agencies regarding ongoing compliance with the SRD [by September 2023].

A process will be developed to make the change procedures of the survey cadence, scheduling, and strategy more visible and inform the community and agencies regarding ongoing compliance with the SRD.

PREOPS-3527

SP-III.3-10

Add a formal review and sign off following pilot processing runs

JOR_2023_02

2024-02-01

Not Started

Add a formal review and sign off for the transition to data release processing. The signoff should occur between the end of the pilot run processing and the start of production processing [before the next review].

A formal review and sign off following pilot processing runs will be added

PREOPS-3526

SP-III.3-9

Mitigate system engineering work overload

JOR_2023_02

2023-12-31

In Progress

Mitigate the upcoming system engineering work overload in the handoff between Rubin construction and RPF operations. One possible mitigation is the use of external contractors [by the end of 2023].

Consulting help will be hired to mitigate system engineering work overload

PREOPS-3525

SP-III.3-8

Implementation of the Operations CCB

JOR_2023_02

2023-12-31

Not Started

Complete the implementation of the RPF CCB and RB [by the end of 2023].

The Operations CCB will be implemented by the RPF Systems Engineering team. The Systems Engineering team has few resources to dedicate to pre-operations work so consulting help will be needed

PREOPS-3524

DM-III.2-7

Metric of success

JOR_2023_02

2024-06-28

Not Started

Define a metric of success that enables the RDM team to evaluate their chosen user support model, and the efficacy of the community support [by Data Preview 1].

One or more metrics of success will be defined to enable evaluation of the chosen user support model, and the efficacy of the community support

This recommendation  is specific to the user support model. I suspect that the reviewer mistakenly put RDM when it should have been RPF.

PREOPS-3523

DM-III.2-6

Concise and complete set of performance metrics

JOR_2023_02

2024-06-28

In Progress

Develop and agree on a concise and complete set of performance metrics that are tracked by the team and reported to operations management [by Data Preview 1].

We do need to document a set of (non science science) performance metrics - it has not been done yet.

[~yusra] [~lguy]   do we need a joint ticket to start a non science metrics document - or paste some idea here i will start it ..

PREOPS-3522

DM-III.2-5

Agreement with SLAC

JOR_2023_02

2023-06-01

In Progress

Re-evaluate the agreement with SLAC offering new personnel software staff positions and consider instead continuing existing software staff that could transition from the NSF construction project [by June 2023].

The FY24 Scrub allowed us to explore this topic and make some advances. SLAC has been working with us to retain more staff from construction. It not obvious we can re-valuate the agreement with SLAC, DOE need to fund half the project. We will keep pressing our case.

I am ok with the progress on this. I think we have made progress. Beyond FY24 there is still detail scrubbing to do. Do we close this out for now. I think so. Will approve and Wil can reopen if needed.

PREOPS-3521

DM-III.2-4

Decision to split software developer

JOR_2023_02

2024-02-01

In Progress

Re-evaluate the decision to split software developer management between different departments [by the next operations review].

We will convene a discussion among the ADs to see how to best move forward.

We will convene a discussion among the ADs to see how to best move forward.

PREOPS-3520

OO-III.1-3

Plan for SLAC maintenance

JOR_2023_02

2023-10-01

Not Started

Ensure that the plan for SLAC maintenance of the LSSTcam is fully described for future reviews [by FY 2024].

Both the construction and operations teams (they are essentially one in the same) continue to work closely with the team at SLAC developing LSSTCam. Maintenance plans are continually being updated as specific details are learned during the final test and verification action phase for the instrument. These details are being folded into the Rubin Observatory Operations (ROO) strategic maintenance plan and include reassessment of both technical and personnel resource needed to maintain LSSTCam through the 10-year LSST program. The ROO Strategic Maintenance PLan do men tis a key deliverable for FY23 and will be a featured discussion point in subsequent reviews.

Both the construction and operations teams (they are essentially one in the same) continue to work closely with the team at SLAC developing LSSTCam.  Maintenance plans are continually being updated as specific details are learned during the final test and verification action phase for the instrument.  These details are being folded into the Rubin Observatory Operations (ROO) strategic maintenance plan and include reassessment of both technical and personnel resource needed to maintain LSSTCam through the 10-year LSST program.  The ROO Strategic Maintenance PLan do men tis a key deliverable for FY23 and will be a featured discussion point in subsequent reviews.

PREOPS-3519

OO-III.1-2

Re-consider the on-site presence

JOR_2023_02

2023-10-01

Not Started

Re-consider the on-site presence of engineering and technical support staff during nighttime for the first years of operations, until steady-state operation with nominal unplanned downtime has been reached [by FY 2024].

As the construction project progresses through its commissioning phase it is clearly evident that significant technical presence will be needed in the early stages of operations. The Rubin Observatory Operations (ROO) plan as presented is for steady state with a minimal technical presence needed at the summit. We have always planned that there would be a transition period between the end of construction and steady state operations. The details of the needed technical staff during nighttime operations is becoming clearer as the construction commissioning advance. Adjustment are currently being made in the detailed transition plan for nighttime support.

With the need for Technical ical support in mind we are also training up the nighttime Observing Specialists to have some level of technical awareness and expertise to server as the front line for diagnosis and solving technical issues as they arise. This is a core part of the Observing Specialists functions and has been part of the plan from the beginning. The technical training of the Observing Specialists is part of the designed work scope during construction commissioning.

As the construction project progresses through its commissioning phase it is clearly evident that significant technical presence will be needed in the early stages of operations.  The Rubin Observatory Operations (ROO) plan as presented is for steady state with a minimal technical presence needed at the summit.  We have always planned that there would be a transition period between the end of construction and steady state operations.  The details of the needed technical staff during nighttime operations is becoming clearer as the construction commissioning advance.  Adjustment are currently being made in the detailed transition plan for nighttime support.

With the need for Technical ical support in mind we are also training up the nighttime Observing Specialists to have some level of technical awareness and expertise to server as the front line for diagnosis and solving technical issues as they arise.  This is a core part of the Observing Specialists functions and has been part of the plan from the beginning.  The technical training of the Observing Specialists is part of  the designed work scope during construction commissioning.

PREOPS-3518

OO-III.1-1

Monitor the effectiveness of the initial maintenance strategy

JOR_2023_02

2023-10-01

Not Started

Monitor the effectiveness of the initial maintenance strategy during the first 2 years of operation and adjust the strategy accordingly if needed. Reserve corresponding resources in the operations plan [by FY 2024].

As part f the Rubin Construction integration, test and commissioning effort the Rubin Observatory Operations (ROO) team its carefullym monitoring the maintenance needs of the observatory. As systems are bing integrated and tested the ROO team ias part of their construction/operations dual responsibilities are documenting technical issues and noting specific maintenance needs. These will be recorded in the Computerized Maintenance Management System (CMMS) under development by the construction project. This body if information is currently being used to update the maintenance strategy and planning with resultant resources needed on various timescales.

The ROO Strategic Maintenance Plan is a key deliverable from the FY23 early operations efforts. This document is being develop[ed now with contributions from the ROO Team and Group leads (who are also part of the Rubin construction team). Staffing and non-labor resource adjustments are expected from the Maintenance Strategic Plan update and will be part of revised operations planning for ROO in FY24.

As part f the Rubin Construction integration, test and commissioning effort the Rubin Observatory Operations (ROO) team its carefullym monitoring the maintenance needs of the observatory.  As systems are bing integrated and tested the ROO team ias part of their construction/operations dual responsibilities are documenting technical issues and noting specific maintenance needs.  These will be recorded in the Computerized Maintenance Management System (CMMS) under development by the construction project.  This body if information is currently being used to update the maintenance strategy and planning with resultant resources needed on various timescales.

The ROO Strategic Maintenance Plan is a key deliverable from the FY23 early operations efforts.  This document is being develop[ed now with contributions from the ROO Team and Group leads (who are also part of the Rubin construction team).  Staffing and non-labor resource adjustments are expected from the Maintenance Strategic Plan update and will be part of revised operations planning for ROO in FY24.

Fri Jul 28 00:02:36 UTC 2023