Cycle 19

There was no formal call for Cycle 19 proposals. Observing of the current Long Term project portfolio will be continued in Cycle 19, 1 Dec 2022 – 31 May 2023. All Priority A allocations already made will be honoured; the time designated as look-ahead will then be taken up in Priority B (i.e., on a best-efforts basis, subject to scheduling and logistical constraints, but there is likely to be a significant amount of Priority B observing possible). This will ensure strong scientific productivity will be maintained. The Director’s Discretionary Time (DDT) mechanism will operate in Cycle 19 as usual (with an enhanced amount of available time): it will provide a continuous entry point for some smaller projects, from any proposer(s), who will need to demonstrate both scientific merit and urgency, to be scheduled on a best-efforts basis.

Instability of the Poznan and Juelich LTA sites

Data staging and downloading data from the Poznan and Juelich LTA sites have been intermittently unstable in the past few months. This is receiving the needed attention and will be resolved as soon as possible. Contact SDC-Helpdesk if you experience any issues.

LOFAR password self-service

The password Self-service is active again although the integration with (linking from) MoM/NorthStar/LTA is not yet functional. In the meantime, users can go to the link: https://webportal.astron.nl/pwm to reset their password. Contact SDC-Helpdesk if further support is needed.

LOFAR Status Meetings (LSM) survey

Please fill in the survey here for the preferred format and focus of LSM. The deadline is November 4, 2022.

• Currently, there are 38 stations operational in the Netherlands: 24 core and 14 remote stations. 14 international stations are in operations: DE601, DE602, DE603, DE604, DE605, FR606, SE607, UK608, DE609, PL610, PL611, PL612, IE613, LV614.
• A new international station will be built in Italy. Station deployment is planned to start in 2022. The station is expected to be fully operational in 2024.

• Antenna elements of all stations are performing nominally. The overview of non-operational antenna elements for LBA and HBA is available here. At the stations level, a few stations including DE604 and DE605 have had their air-conditioning units replaced in the past few weeks. PL610 has been offline for almost a month with a broken network switch. The replacement has been installed last week.
• An unplanned power outage in the housing of COBALT2.0 at CIT in Groningen resulted in the breakdown of one of the switches of the cluster causing the observing system to be offline for a couple of days in September.
• A major upgrade of the CEP processing cluster is planned in 2023.

• Cycle 18 observations started on 1 June and will run till 30 November 2022. The observing schedule can be found here.
• At the end of observing week 20, 85% of the observing program has been completed, with an average observing efficiency of 67%. About 270 observing hours (representing 8% of production hours) have been lost due to high temperatures, lightning, active Sun, COBALT switch breakdown, and other system issues.

Telescope Manager Specification System (TMSS) - S. ter Veen

• The Telescope Manager Specification System (TMSS) is a replacement for MoM and various schedulers to specify and schedule the LOFAR observations.
• TMSS has been running part of the observing program since April 25th. We have done over 500 observations now with TMSS.
• While observations have been running stable, there have been issues with the pipelines and the ingest completeness. This has delayed the delivery of some of the data. The issues with the pipelines are now fixed. Further fixes for the ingests are ongoing. Fixing the production observations is ongoing.
• Commissioning is ongoing to accept other observing strategies and functionality in production. The goal is to run Cycle 19 entirely with TMSS
• Users will be instructed on how to use TMSS when their project is moved over to the new system.
• Development for the last two months focused on:
  • Delivery of dynamic scheduling.
  • Delivery of project and cycle reports.
  • Update of front-end software components.
• Development funding has been extended till maximum the end of December to make sure that TMSS is production ready and that the critical bugs are resolved.
• Further development of TMSS will continue next year under the LOFAR development program together with the other LOFAR2 development projects.

LOFAR2.0 program – W. van Cappellen

LOFAR2.0 progress highlights are presented every two months in the LOFAR Development Newsletter, see: https://www.astron.nl/lofar2-0-newsletter/.

• Over the past summer we have been using the Dwingeloo Test Station (DTS) at ASTRON intensively. We have made use of some extremely warm days to investigate the thermal and cooling properties, we have done tests to determine the noise contributions originating in our processing subracks, in the cabinet, RF signal leakage through the cabinet walls, we looked at cross-talk between input channels on a single RCU2, and between different RCU2s, we managed to calibrate the cable delays from the test field antennas and correct for that and, ultimately, we have been able to show that digital beamforming is working, which is a highly important milestone for the project.
• As a next step, the DTS hardware was moved to the LOFAR2.0 Test Station (L2TS), formerly known as CS001. The L2TS provides the RFI-quiet environment we need to run our final tests for sensitivity and linearity. The CS001 station is positioned conveniently close to the road and the concentrator-node building near the ‘Superterp’ at Exloo. It is a core station, therefore with two HBA fields. This allows us to experiment with ‘long baselines’ in a single station. Next to these activities, we have ordered additional hardware to make L2TS a fully equipped LOFAR2 station, the first of its kind. We expect to have the full L2TS ready by the end of quarter one of next year, 2023.
• Over the summer, we have already used CS001 in its former LOFAR1 configuration to create a collection of reference LOFAR1 datasets, to which we can compare the upcoming LOFAR2 datasets.
• The EU public tender to manufacture the LOFAR2.0 station hardware has been concluded. The purchase orders to produce all LOFAR2.0 hardware will be placed in the first half of November.
• Phase 1 of the DANTE project, aiming to develop a replacement of the HBA-FE board, successfully passed its Preliminary Design Review. The next step is to build an engineering model on which all components (LNA, delay lines, filters, control) are integrated.
• We chose to delay the replacement of CEP4 by CEP6 by one year to achieve a better alignment with other activities (LOFAR2.0 deployment, replacement of the central network infrastructure, COBALT3.0 definition, and the LOFAR ERIC formation). The support contact of CEP4 has been extended by 1 year.

SDC program – J. Swinbank

• A major goal of the SDC development effort over the last several months has been to enable pipeline processing as part of the LDV project. Most recently, this work has concentrated on the development of a web application — “LDV Spec” — that is used to select and specify datasets for processing. We are aiming to deliver this functionality in quarter 4 of 2022; at this point, the major development effort to deliver bulk LDV processing will be complete, although further work will continue to support, maintain, and extend the LDV processing system. It is important to note that the technologies developed to support LDV are not ultimately intended to be project specific; rather, bulk processing in the LDV project demonstrates an important new capability that will be deployed in support of a range of SDC activities and services over the next several years.
• In parallel with the LDV effort, work continues to release a first version of the Rapthor pipeline. Rapthor will provide direction-dependent calibration and imaging for LOFAR data. Together with LINC — released earlier this year and providing direction-independent calibration — this will provide a standardized, supported imaging solution to SDC operations and other stakeholders, and provide a solid basis for ongoing development in preparation for LOFAR2.0.
• Over the last several years, the SDC development team has played a leading role in ESCAPE, an EC-funded project to develop common infrastructures for data handling and analysis across major European research infrastructures. The current ESCAPE project will end in January 2023 (although future collaborations are planned). The project's close-out event, ESCAPE to the Future took place in Brussels in late October. At this event, the project team presented an overview of the work carried out in ESCAPE and provided a vision for the future of scientific data infrastructure — a vision we hope to put into practice over the next several years. Although it may be too late to participate by the time this newsletter is released, workshop materials should be made available online after the event. The group picture taken at the event is shown here.

• When the SKA comes online later this decade, it will not provide data products directly to end users but rather to a global network of “SKA Regional Centres” (SRCs) which will be responsible for archiving the data, providing it to the community, and supporting that community in delivering high-impact scientific results. Development of that SRC network is now moving rapidly. The SDC team is collaborating with colleagues worldwide in defining the overall technical architecture of the network, with a particular emphasis on the “science platform” which will be the primary analysis interface presented to astronomers. We collaborated with the NL SKA Office to host an initial “community meeting” to introduce and discuss the SRC concept to our local scientific community in September; we expect this to be a first step in a wider effort to bring together the technological and the scientific aspects of the project. This may seem rather distant from LOFAR news, but in fact we expect them to be closely related: our experience with LOFAR educates and informs the work we can do for SKA, while the efforts of the (much larger!) SKA development community will directly feed back into the services we are able to provide for LOFAR.

LOFAR Data Valorization – M. Iacobelli & C. Baldovin (on behalf of the LDV project team)

• As anticipated in the previous newsletters, the LDV project has entered a new exciting phase: the data compression pilot phase in October2022. With this ongoing effort the LDV team aims to: decrease the occupancy of beamformed (BF) data on tape at SurfSara, correct and reconstruct missing metadata in the data collection catalogue, test and fine tune the operational procedures for the upcoming full operational phase.
• During the pilot phase 2967 BF datasets distributed in more than 20 science projects will be edited, amounting to an overall size of 0.96 PB [see in the images the characterization of the data to be processed during the pilot phase]. Debugging is also an important outcome of this pre-operational phase. Improvements to the robustness of the workflow were highlighted during the past weeks.
• The overall result of the pilot will enable the LDV team to submit a proposal to extend the usage of computational resources at SurfSara, thus enabling the compression of the interferometric data in the LTA.
• In the coming weeks we will be moving towards an operational regime to complete the first stage of the processing of BF data. At the same time, the development will continue during the next months to enable the next step; which is the re-quantization of BF data. We expect that this step will further release close to 4 PB of storage in the LTA.

• Course on “Frontend research at low radio frequency Radio astronomy: Science and technical challenges”, l’Aquila, Italy, 3-7 April, 2023. More information here.