Cycle 20 allocations

A total of 42 single cycle observing proposals were received at the end of the 8 March 2023 proposal call deadline. Of those, 38 were successful. Only 4 proposals couldn’t make the mark.

Here is the summary of the allocations of the successful Cycle 20 projects:

• Observing period: 1 June 2023 to May 31, 2024
• Requested observing time: 16905.73 hours.
• Awarded observing time: 9338.0 hours.
• Awarded processing time: 3673.9 hours.
• Awarded LTA storage (Dysco): 4142.5 TB
• Awarded support time: 1058.9 hours.

The awarded resources exclude provisions for DDTs and commissioning projects. The observations are being scheduled using the dynamic scheduling functionality of the Telescope Manager Specification System (TMSS).

LOFAR family meeting 2023

This year’s LOFAR family meeting happened in Olsztyn, Poland, from 12 to 16 June as scheduled. Here is the link to the programme. There is a plan to make the talks public at a later date. Users will be notified when the talks are online.

Data staging and downloading from the LTA sites

Data staging and downloads from the LTA sites were stable in the past few weeks. Contact SDC-Helpdesk if experiencing any issues with staging and downloads.

Call for LOFAR2.0 large programmes

The deadline for the call is noon (UTC) of October 12, 2023. An overview and capabilities of LOFAR2.0 can be found here. Details of the call may also be found at this link.

• 38 stations operational in the Netherlands: 24 core and 14 remote stations. 14 international stations in operations: DE601, DE602, DE603, DE604, DE605, FR606, SE607, UK608, DE609, PL610, PL611, PL612, IE613, LV614.

  New international stations will be built in Italy and Bulgaria in the coming years. Both stations will be equipped with LOFAR2.0 hardware, whose installation/rollout is expected in 2025.Antenna elements of all stations are performing nominally, except, RS503 HBA that has about 17% non-operational elements. Those elements are either broken to be fixed or broken beyond repairs. The overview of non-operational antenna elements for LBA and HBA is available here. At the stations level, all stations are online during the period under consideration, except, IE613 which was out for few days in early June due to power supply/connection issues caused by storms at the station.

• Moreover, no major failures occurred on CEP hardware/software over the past couple of months. A major upgrade of the processing cluster is planned to happen later this year or early next year.

• Cycle 20 observing campaign started on 1 June 2023 and will run till 31 May 2024. The public version of the observing schedule in TMSS will be available soon. For now, only PIs can see their projects in TMSS.
• At the end of the first two months, we have done more than 550 hours of production observations.

LOFAR Development program (W. van Cappellen)

• The hardware of the first full LOFAR2.0 Test Station (L2TS phase 2) has been integrated in the lab. Verification tests of the system revealed several issues that could be easily solved. The hardware is installed in CS001 on July 5 and 6 for further verification and validation.
• The upgrade of the central network switches in Groningen has been approved. With the upgrade, based on 400 Gbps technology, the network will be ready for LOFAR2.0 and even higher data rates from the stations to the central processing systems in future.
• The White Rabbit switches for the Timing Distributor upgrade have arrived (see Figure 1). This year, most of the Remote Stations are upgraded to this extremely accurate clock distribution system. The Core Stations follow in 2025.
• The DANTE project is testing the first integrated prototype of the new HBA frontend electronics board.
• Do you want to know more? LOFAR Development Newsletter.

SDC program (J. Swinbank)

• First release of the Rapthor pipeline

  Rapthor is a pipeline to process LOFAR data and produce high-quality images. Specifically, it performs direction-dependent calibration in a fully automated self-calibration loop.

  The accompanying image (Figure 2), produced by Rapthor, has a maximum colour scale of 3 mJy/beam, so many sources that are of sub-mJy/beam brightness are visible. It is made from 12 MHz bandwidth of an HBA observation, and reaches an expected noise level of 150 µJy, which for such an integration time and bandwidth is approximately the best that can be achieved with LOFAR NL.

  In the future, LOFAR users will be able to directly request the high-quality images from Rapthor when proposing observations, or they can run (and modify) the pipeline for themselves.

  Rapthor uses modern design and engineering principles and, where possible, is built around off-the-shelf tooling like the Common Workflow Language. This makes it easy to maintain and support development, and to reuse parts of the pipeline for different science cases.

  Development of the pipeline is helping to push our existing processing software (e.g. DP3, WSClean, PyBDSF) to new limits, and these improvements are already eagerly used by various (expert) users across the LOFAR Community.

  By releasing our software, we are telling the community that the pipeline is ready to be used by others. You can get started by taking a look at the release notes [1] and the documentation [2]. Our current release targets the processing of Rapthor HBA NL data, but this is just the beginning — in the future, this will be extended to support long baseline and LBA data and to make use of all LOFAR2 capabilities.

  [1] https://rapthor.readthedocs.io/en/latest/changelog.html
  [2] https://rapthor.readthedocs.io/en/latest/

• Data processing systemDevelopment continues apace on the data processing system that is being built to facilitate at-scale processing of data stored in the LOFAR LTA. This work is currently focused on providing support to the LDV project, but will grow to provide us with the capability to process data on behalf of all users - subject to resource availability - in the LOFAR2.0 era.
• Upgrades to stagingStageIT, a replacement for the existing LTA staging service that will bring improvements to both usability and reliability, is in its final stages of development and has already been used by some users in an early-access mode. We are still refining the user experience and ensuring full backwards compatibility with the old system, but we look forward to putting this into widespread operation soon.
• Planning for the LOFAR2.0 eraThe recent call for LOFAR2.0 Large Programmes was accompanied by a release of documentation on LOFAR2.0 and its supporting systems [1]. Amongst these was a discussion of the data management capabilities that will be provided through the ASTRON Science Data Centre and the other ILT partners in the LOFAR2.0 era [2]. We presented more information during the “LOFAR2 & Users” at the LOFAR Family Meeting in Olsztyn [3]. Taken together, we hope this material presents a useful and compelling picture of our ambitions for the future, and we welcome feedback from the widest possible community.

  [1] https://www.lofar.eu/lofar2-0-documentation/
  [2] https://www.lofar.eu/wp-content/uploads/2023/06/Data_Management_Capabilities.pdf
  [3] http://lfm2023.uwm.edu.pl/

Telescope Manager Specification System (TMSS; A. Schoenmakers, S. ter Veen)

• One of the major challenges for the LOFAR support staff is to fill the LOFAR telescope calendar efficiently with approved observation programs. The scheduling process has to take into account many parameters, such as project priority, source visibility, antenna availability, maintenance activities like mowing and repairs and many more, sometimes unexpected, events. Needless to say that this puzzle creates a heavy burden on the Observatory staff responsible for carrying out the observing program. To alleviate that burden, automating the scheduling process has been one of the major requirements for TMSS. The functionality for this was named the "Dynamic Scheduler". Dynamic, because it allows an operator to rerun the scheduling process whenever a constraint changes. This can be a station dropping out, a trigger caused by an event that requires LOFAR to observe that target of opportunity, or some other event that changes the capability of the telescope.
• After a long period of extensive testing, the important decision was made to start using TMSS fully, including the dynamic scheduler, for LOFAR observing cycle 20 which started on June 1st. This decision is a major milestone for the TMSS team and the dynamic scheduler.The Dynamic Scheduler will relieve the burden on everyone involved in the day-to-day planning of LOFAR observations. The first production period, starting on June 1st, helped us to iron out a few remaining issues. Overall it works well without any showstoppers and the supporting staff is getting used to it quickly. A bright future lies ahead!
• In the figures we show some of the plots that are part of the automated scheduling process. They show a day-schedule for LOFAR, resulting from automatically scheduling a batch of observations, and a few plots showing the 'goodness of fit' of some of the scheduled observations, with respect to the constraints that these observations should adhere to.With TMSS and the Dynamic Scheduler now in full production mode, we are looking forward to extend the scheduler to include other, new constraints such as, for example, the state of the ionosphere which has a major impact on the data quality of a low-frequency radio telescope such as LOFAR (comparable to optical seeing for a visible-light telescope). With TMSS we are looking forward to improve the usability and responsiveness, and to include all requested observing types of Cycle 20.

The dates of LOFAR Status Meetings, roll-outs and stop days are listed in an online calendar that is available here. In particular, we emphasize:

• Next LOFAR newsletter: October 2023
• 7th LOFAR Data School: April 2024