RTN-064
JOR recommendations#
Review recommendations extracted from Jira.
2026-09-22 16:21
Issue key |
Rec# |
Summary |
Report date |
Due Date |
Implementation Status |
Assignee |
Description |
Response |
Implementation Status Description |
|---|---|---|---|---|---|---|---|---|---|
Complete and Validate Integration of Nightly Observing Quality Assessment Tools |
CCR2_ORR1_2025 |
2025-09-30 |
Manager Approved |
Leanne Guy |
NSF and DOE: Complete and validate the integration of the nightly observing quality assessment tools. |
Nightly validation tools are in place as part of the plot navigator : [+https://usdf-rsp.slac.stanford.edu/plot-navigator/+|https://usdf-rsp.slac.stanford.edu/plot-navigator/] They have been tested during commissioning and are ready for use with the LSST. The QA tools will continue to evolve over the course of the survey. |
|||
Update Scheduler to Final LSST Survey Configuration and Resolve SV Survey Faults |
CCR2_ORR1_2025 |
2026-12-31 |
In Review |
Leanne Guy |
Update the scheduler to final configuration of LSST survey based upon survey optimization and potential changes in exposure cadence. Resolve any scheduler faults that may have occurred during the SV survey. Complete before the end of CY 26. |
Version 5.3 of the SCOC recommended survey strategy has been approved. The version of the cadence was in the process of being implemented when the July 2026 storm hit Cerro Pachón. It will be fully implemented when we return to sky. |
|||
Convene Data Release Board to Review Potential Revisions to DP2 and DR1 Schedules |
CCR2_ORR1_2025 |
2025-09-22 |
Manager Approved |
Zeljko Ivezic |
Convene the Data Release Board before the beginning of shutdown in September 2025 to decide whether there should be revisions to the DP2 and DR1 schedules. |
In progress (see charge in RTN-052) |
Work is in progress, including both science collaborations, various boards and team members, to (re)define DR1 and DR2 contents and time scales. We expect resolution by the end of CY 2026. |
||
Enhance Active Optics Performance by Increasing Team Staffing and Allocating Regular Engineering Nights for Tuning & Calibration |
CCR2_ORR1_2025 |
2025-12-31 |
Manager Approved |
Kevin Reil |
Increase the staffing for the team, improving active optics performance (including, if possible, a controls engineer) in time to maintain current staffing levels. Consider allocating dedicated engineering nights regularly for Active Optics tuning and calibration work. |
We are reallocated some of the staff to AOS in particular and we are planning to prioritize Image Quality improvement tests until the Rubin Obs project agrees the performance is useful for survey data taking. |
This response/implementation of this recommendation is complete. Resources have been added and progress has been made. Further progress can be tracked here. [+rso-9+|https://rubinobs.atlassian.net/browse/RSO-9]. |
||
Update Telescope & Site Risk Registry with Detailed Items and Separate Mitigations for Ops Transition |
CCR2_ORR1_2025 |
2025-12-31 |
Manager Approved |
Holger Drass |
Update the Telescope & Site risk registry by Dec. 2025 to include details for the current risk items and prepare the registry for transition to Operations. The Telescope & Site risk registry is generally good but lacks details specific to the most likely and concerning risk items related to completion of the dome tasks, and it groups together risk items with different mitigation plans that should be tracked separately |
We need to update the risk register |
The Rubin Scrub is now ongoing, including a complete overhaul by the risk owners. As part of the process, we have detailed several risks. For example, in the cooling system, we have identified risks for each type and are updating the corresponding mitigation plans. With over 200 risks we have assessed, most of which are active and actively mitigated, we have reached a high level of detail. Closing here, noting that Further detailing will be part of the ongoing risk management process as needed. |
||
Finalize Image Quality Optimization with AO Controls Expert |
CCR2_ORR1_2025 |
2026-12-31 |
In Progress |
Sandrine Thomas |
NSF responsibilities: Before the end of CY 26, complete the optimisation of the image quality with the help of an external expert in AO controls (e.g., Marcos Van Dam). |
We have asked Luke Johnson from NSO to review some of our issues. We are also planning to seek help from other AO experts |
The team is working on two aspects of the Active Optics System:
As we are getting more and more access to the summit after the storm, the team is pushing to install two more louvers actuators over the RAD doors as well as thermal sensors on the TMA. |
||
Finalize Outstanding Science Requirements Validations |
CCR2_ORR1_2025 |
2026-12-31 |
To Do |
Zeljko Ivezic |
NSF and DOE: Complete the outstanding Science Requirements Validations before the end of CY 26. |
It will be addressed once more data are collected and analyzed. |
The validation team will complete the outstanding Science Requirements Validations once we are collecting on-sky data again. |
||
Prioritize and Complete ARC Flash Study with Electrical Panel Labeling |
CCR2_ORR1_2025 |
2025-10-11 |
Manager Approved |
Juan Fabrega |
ARC Flash study, including labeling of all electrical panels needs be high priority and completed [by CCR 3] |
An external company called Julnor carried out a preliminary audit, in which the areas and electrical equipment were assessed. It did not carry out a more detailed assessment, but it did provide a series of recommendations that must be complied with in order to comply with current regulations. |
A final report has been delivered. |
||
Develop a walk-up plan for the NSF FY26 funding scenario |
CCR2_ORR1_2025 |
2025-09-30 |
Manager Approved |
Robert Blum |
Develop a walk-up plan for the NSF FY26 funding scenario by September 30, 2025. |
We are doing this as part of our ongoing budget planning. |
A 10% budget reduction scenario has been developed. The walk up is to use carry forward funding to cover up to a 10% reduction of NSF funding in FY26. |
||
Expand Risk Register to Include Project-to-Operations Risks at Granular Level |
CCR2_ORR1_2025 |
2025-10-21 |
Manager Approved |
Holger Drass |
Expand the risk register to include risks that can flow down from the Project to the Operations, at a sufficiently granular and specific level (before CCR3/ORR2). (See comment above. These should be captured by the Risk Board) |
This recommendation is complete. The Rubin Risk Manager has worked to capture construction risks in operations as appropriate. The risks are included in the new NOIRLab JIR Risk tool (developed with input from Rubin). The risks captured from cosstruction will now be managed forward like all the Rubin Operations risks. |
|||
Formalize Regular Meetings Between Operations PMs and Stakeholders (Including S3DF) to align on Rubin Requirements and Work Planning |
CCR2_ORR1_2025 |
2025-09-30 |
Manager Approved |
Phil Marshall |
Formalize regular meetings between Operations Program managers and all relevant stakeholders, including S3DF, to ensure proper communication of Rubin requirements and planning/execution of related work by the end of September 2025. |
We are doing this. A monthly series of one-on-one meetings between the Deputy Director at SLAC and the SCS Division Director and S3DF Manager was begun in August, and the first two meetings have been held. |
Two series of monthly meetings have been established between the Deputy Director at SLAC and two key stakeholders, the S3DF Manager and the DOE Program Officer. We will build on this with further series as the need is identified. |
||
DM-2026:1-1 |
Revisit DR1+ Release Schedule & Cadence |
JOR_2026 |
To Do |
Zeljko Ivezic |
The Rubin Data Release Board should revisit the data release schedule and cadence beyond DR1, with a clear inputs and procedure defined, and make recommendations to the Rubin Director. [deliver and review before the end of FY27] |
JOR Report (SC1, Comments 10) — The Rubin Data Release Board should revisit the data release schedule beyond DR1, especially related to the cadence. This process will require close collaboration with the data management team and the science collaborations, and should consider the quality and performance of the existing data, based on science metrics, and take into account labor and hardware resource costs of the production campaigns. A clear procedure should be defined that leads to a timely decision on the release schedule and cadence by the Rubin Director. |
|||
DM-2026:1-2 |
Strategic Hardware Procurement Plan |
JOR_2026 |
To Do |
William O’Mullane |
Produce and maintain a strategic hardware procurement plan. [deliver and review before the end of FY27] |
JOR Report: (SC1, Comments 11) — Given the challenging environment around computing costs, produce a strategic hardware procurement plan that states unit-cost assumptions (CPU, potentially GPU, SSD/spinning disk, and tape) explicitly and reconciles them annually against prices actually paid, carries at least three price scenarios including flat and rising, attaches a named descope or deferral option to each scenario, and addresses the equipment refresh liability beyond FY33 that the current funding profile does not appear to cover. Impacts of potential release schedule changes should be folded into this strategy to come up with a cost-effective plan. |
|||
DM-2026:1-3 |
Establish Rubin AI Strategy Group |
JOR_2026 |
In Progress |
Phil Marshall |
Establish a Rubin AI Strategy Group to develop a strategic vision and prioritized recommendations for the adoption of artificial intelligence and machine learning across Rubin operations. [group constituted by December 2026; initial vision by June 2027, in time to inform FY29 procurement commitments; full recommendations by December 2027] |
We will do this, using the JOR2026 SC1 report Comment 12 to charge the AI Strategy Group. |
We will do this, using the JOR2026 SC1 report Comment 12 to charge the AI Strategy Group. |
||
DM-2026:1-4 |
Assess Feasibility of Smoothing DM Staffing |
JOR_2026 |
To Do |
William O’Mullane |
Rubin and the agencies should jointly assess the cost and feasibility of smoothing the operations staffing profile through DR1 delivery. [staffing process before the end of FY27] |
JOR Report: (SC1, Comments 1) — The committee commends the Rubin team for the successful release of the EDP2 and for supporting the community that showed great interest in this public data. Overall, the Data Management team is operating effectively. Science performance meets or exceeds every design specification presented, the multi-site processing architecture has been settled through deliberate engineering judgement including the replacement of a workflow system found to be unstable, and the DOE funded USDF concept of operations has matured substantially. The committee has no serious concerns about the operation, but there are concerns regarding budget assumptions and its staffing trajectory. [Note - this comment may not be 100% relevant] Note: This comment may not be 100% relevant - found using AI. |
|||
DM-2026:1-5 |
Re-baseline Release Schedule vs. Staffing |
JOR_2026 |
To Do |
William O’Mullane |
Rubin should re-baseline the data release schedule against the staffing actually projected to be available. [schedule re-baseline before the end of CY2026] |
JOR Report: (SC1, Comments 2) — DM tracks extensive metrics but little of it appears in the management plan, which mentions performance indicators as an intention and defines none. None of these measures are expressed as service-level commitments. Targets exist for science quality and forum response, but not for the availability, latency or resolution times of the services on which the community will depend after DR1. The committee is worried that the gap between what the team does and what its plan records will become an issue as the DM team’s staffing and institutional memory declines. The alert requirement is not currently met. The team measures this, attributes the failure to a specific cause and has hired against it, and the requirement does not yet formally apply. Nonetheless the entire failure budget is consumed before network or embargo outages are counted. The committee notes that RTN-046 describes DM as working normal business hours with no guaranteed out-of-hours support. The plan describes on-call arrangements in conditional terms and acknowledges they may be difficult to formalise institutionally. The requirement and the support model behind it should be brought into alignment before the requirement takes effect. [Note - this comment may not be 100% relevant] Note: This comment may not be 100% relevant - found using AI. |
|||
DM-2026:1-6 |
Discuss DM Staffing/Competencies with NSF & DOE |
JOR_2026 |
In Progress |
Phil Marshall |
Rubin management should discuss current staffing levels and competencies of the DM team with NSF and DOE, to address concerns related to hires and assignments resulting from the NOIRLab and SLAC matrix structure [before the end of FY27] |
We will do this. We accept the key input from the JOR2026 report Comments that “where the Rubin team carries accountability for a deliverable, it should have a defined role in selecting the staff assigned to meet it, including participation in the assessment of candidates put forward by contributing institutions.” We will establish this as our process for staffing open positions. |
We will do this. We accept the key input from the JOR2026 report Comments that “where the Rubin team carries accountability for a deliverable, it should have a defined role in selecting the staff assigned to meet it, including participation in the assessment of candidates put forward by contributing institutions.” We will establish this as our process for staffing open positions. |
||
DM-2026:1-7 |
Engage Developers to Accelerate Code / GPU Transition |
JOR_2026 |
To Do |
William O’Mullane |
Rubin DM should engage additional expert developer(s) to facilitate the acceleration of the most time-consuming codes, including partial transition to GPU’s. [by end FY27] |
JOR Report: (SC1, Comments 3) — Rubin, like everyone else, faces significant hardware cost increases. Storage is a critical component, and the DM team should explore options for use of cloud storage, different SSD and hard disk and tape. Using GPUs is an opportunity with on-going R&D that could lead into HPC usage. The PanDA system is still a back-up solution for DR1. The operations of the new system looks labor intensive and there are opportunities for more automation in running the system. |
|||
SO-2026:2-1 |
Resolve Dome Louver Actuator Acquisition |
JOR_2026 |
To Do |
Sandrine Thomas |
If an adequate resolution of the actuator acquisition is not reached by October 1, 2026, Rubin management should convene a review to consider and implement an alternate approach. |
JOR Report: (SC2, Comments: Observatory) — Rubin construction continues through FY27. With the start of the LSST survey it is becoming ever more urgent to complete the dome louver automation. Rubin Construction Management should work with the agencies to insure funds are available to complete the remaining punch list items |
|||
SO-2026:2-2 |
Camera Vacuum Leak Strategy & Expert Panel |
JOR_2026 |
To Do |
Kevin Reil |
Develop a list of strategies and impacts for addressing the vacuum leak in the camera. Then convene an expert panel to review options and previous remediation. [by January 2027] |
JOR Report: (SC2, Comments: Camera) — The vacuum leak in one of the feedthroughs inside the camera body is a serious issue albeit currently not affecting camera performance and observing operations. Ongoing leaks can significantly affect the lifetime of the ion vacuum pumps. While the team is evaluating several options to fix the problem during a future long maintenance shutdown, they should actively prepare for a scenario in which the leak worsens significantly requiring immediate action. |
|||
SO-2026:2-3 |
Formalize Issue Escalation To Higher Level Management |
JOR_2026 |
To Do |
Sandrine Thomas |
Rubin management should develop formalized impact thresholds regarding the escalation of the visibility of issues [from the departments] to higher level management. This should include quantitative thresholds based upon cost, FTE effort, and/or impact on observing schedule. Complete by April 1, 2027. |
JOR Report: (SC2, Comments: Observatory) — Ad hoc criteria are used by RSO for identifying significant issues to report to upper management. This can result in inconsistent tracking of RSO decisions that affect observatory and survey performance. |
|||
SO-2026:2-4 |
Continue Image Quality Improvement Plan |
JOR_2026 |
To Do |
Sandrine Thomas |
Continue to execute the Image Quality Improvement Plan. Support the image quality team at the current strength. Report on progress to the agencies by April 1, 2027. |
JOR Report: (SC2, Comments: Image Quality) — Early survey data demonstrated that Rubin can deliver impressive image quality but not yet routinely. RSO stood up a strong team to address the image quality issues. It is important to maintain this team until the image quality requirements are met. An image quality improvement plan was presented outlining short term, medium term, and long-term steps. The committee agrees with the initial focus on the short term activities that have the most significant impact. We commend the ongoing effort to separate and to understand environmental and optics related contributions to the observed ellipticity in the images. |
|||
SO-2026:2-5 |
Support AI in CMMS/FRACAS |
JOR_2026 |
To Do |
Holger Drass |
Support the emerging efforts to implement AI capabilities in CMMS/FRACAS to gain additional operational efficiencies. Report on progress at next annual Operation Review. |
JOR Report: (SC2, Comments: Observatory) — There are emerging opportunities to use AI capabilities to gain operational efficiencies in manual workflows and fault diagnostics. |
|||
ESH-2026:3-1 |
ES&H Corrective Action Plan |
JOR_2026 |
In Progress |
Robert Blum |
Immediately commence work on a corrective action plan for a revised ES&H program and report progress to agencies [by October 2026]. |
Accepted. We will deliver a Corrective Action Plan for safety in Chile in Coordination with NOIRLab leadership and the Rubin Management Board. |
The Rubin Director is working with NOIRLab leadership to establish an action plan for NOIRLab EHS Chile. A draft plan exists and will be presented to the Rubin Management Board for comment. |
||
ESH-2026:3-2 |
Independent External Review of ES&H Program |
JOR_2026 |
In Progress |
Robert Blum |
The Management Board should commission an independent, external review of a revised ES&H program. Focus the review on organization structure, staffing, coherence of the team and recommend corrective actions. Provide the report of the review to the agencies. - [January 2027] |
Accepted. The due date may need to adjust by a month outward to accommodate schedules for the initial Corrective Action Plan which has an internal review in mid October. |
JOR Report: (SC3, Comments (Other) #9) — The previous ES&H reporting structure worked well during Construction. |
||
BS-2026:4-1 |
Hyper-Focus Meetings on Cost/Risk Opportunities |
JOR_2026 |
In Progress |
Phil Marshall |
The Review Committee strongly encourages the team to engage in scheduled hyper-focus meetings to identify opportunities and to develop proactive strategies to realize potential opportunities. The deliberate, hyper-focus on potential opportunities and cost savings and to reduce risks. (Provide initial results to the funding agencies by September 30, 2026) |
We will do this, starting in the run-up to the start of FY27 as recommended and focusing on opportunities to save on cost, reduce schedule and mitigate risk. |
We will do this, starting in the run-up to the start of FY27 as recommended and focusing on opportunities to save on cost, reduce schedule and mitigate risk. |
||
M-2026:5-1 |
Dome Louver Completion Plan |
JOR_2026 |
In Progress |
Victor Krabbendam |
A plan for completion of the louvers should be established by Oct. 1, 2026. If no plan is established by then, a detailed back-up plan (which should be readied now) should be put into motion (by Nov. 15, 2026). |
NSF funding approval has been received. I will be working contract and delivery details over the coming weeks and put updates here. |
|||
M-2026:5-2 |
Formalize In-Kind Contributions (IKC) Expansion |
JOR_2026 |
In Progress |
Robert Blum |
Consider formalizing an expansion of In-Kind Contributions (IKC) to address potential mitigation of DM funding and staff shortfalls (by Feb. 1, 2027). |
We accept this recommendation. We will assess the efficacy of new programs with the IPC team and decide how to move forward. |
We have begun discussions with the IPC team to focus on resource needs. |
||
M-2026:5-3 |
Inter-Lab Cybersecurity Task Force |
JOR_2026 |
To Do |
Robert Blum |
Convene an inter-lab task force to review cybersecurity plans with the intent to harmonize and harden potential security shortfalls. Have a technical review of the above (by September 1, 2027) |
We accept this recommendation. |
JOR Report: (SC5, Comments cont.) — No overarching cybersecurity plan exists for the entirety of the observatory and off-site operations. The cybersecurity plan should include support for critical infrastructure services (e.g., HVAC, electrical, water, etc.,) The lack of a comprehensive cybersecurity plan could invite security breaches that threaten the facility operation and data acquisition. |
||
M-2026:5-4 |
Cybersecurity Tabletop Exercise |
JOR_2026 |
To Do |
Robert Blum |
All stages involved in the data chain should participate in a cybersecurity-focused tabletop exercise (by October 1, 2027) |
We accept this recommendation. |
JOR Report: (SC5, Comments cont.) — No overarching cybersecurity plan exists for the entirety of the observatory and off-site operations. The cybersecurity plan should include support for critical infrastructure services (e.g., HVAC, electrical, water, etc.,) The lack of a comprehensive cybersecurity plan could invite security breaches that threaten the facility operation and data acquisition. |
||
M-2026:5-5 |
Address ES&H Issues Raised in SC3 |
JOR_2026 |
Manager Approved |
Robert Blum |
Address ES&H issues raised in SC3. |
Recommendation accepted. All activity will be captured in the EHS recommendations. For clarity, this recommendation is closed. |
All activity will be captured in the EHS recommendations. For clarity, this recommendation is closed. |
Tue Sep 22 16:21:40 UTC 2026