Event Organisation Guide

Introduction

Welcome to the Software Sustainability Institute’s Event Organisation Guide. The Institute has built up a large amount of experience organising events, ranging from a few to around 100 attendees. This guide goes through the steps of organising an event and is structured around the way the Institute organises its workshops and events.

We use the Collaborations Workshop (CW) series, one of our larger and more complex events, as one of our main examples throughout this guide. The CW series combine audience-led sessions in the style of an ‘unconference’ and traditional workshop sessions. CW has around 80-100 attendees and there is no call for papers but a call for mini-workshops.

Events are Projects

In this guide, we propose to manage events as projects. From that perspective, the stages in a project apply to an event. There are some stages around the running of the events which are of particular importance to give the event context and buy-in from other stakeholders.

In the following sections, we detail each stage to help understand what is needed to plan and organise a successful event.

When to Start Planning

For large events, such as a Collaborations Workshop, it’s important that the idea and feasibility stages start anywhere from nine to twelve months before the event. This is primarily due to the complexity of the event. This will allow enough time for the feasibility stage requirements to be met and the event project stage to have sufficient time to do publicity, sponsorship, registration, calls for participation. This also allows the date to get into the target audience’s diaries before other events are announced.

For smaller events or workshops, e.g. Software and Data Carpentry, three to sixth months are normally enough.

For online webinar type events, one to two months is enough time.

Idea Exploration Stage

At this stage, the event is just an idea. The idea is explored further to see if more investigation is warranted to perform a formal Feasibility stage, which comes next.

An overall goal and aim should crystallise by the end of the Idea Exploration Stage, but this is more formally detailed as part of Feasibility Stage. Think about the target audience before deciding on the goals and aims of the event. An initial understanding of who the audience will be is defined at this stage - i.e. which section of a particular community are you targeting by running this event. This is paramount to develop the next stages of the event: what sessions to run, type of content for each session, publicity, and outputs.

This is also the stage at which the financial parameters should be agreed. For example, will the event make a profit, break even, run as a free event, or is it allowed to make a loss? If the event is allowed to make a financial loss (e.g. a pump-priming event), then the limits of that loss need to be agreed.

You should also agree on the maximum number of attendees at the event as this impacts on venue choices and requirements in the Feasibility Stage.

Even at the Idea Exploration Stage, it should be possible to discuss the balance of the type, or combination of types, of sessions throughout the event in terms of:

• Informing (e.g. speakers)
• Exploring (e.g. discussion session)
• Creating (e.g. hack days, Bring Your Own Data)
• Learning (e.g. Software Carpentry/Data Carpentry or other instructive approaches)
• Networking (e.g. lunch/coffee and other social sessions)

If this stage is approved, an Event Lead should be identified to manage the vision and delivery of the event, and effort from others should be assigned to form a co-ordinating group during the Feasibility Stage. For smaller events, only the EL may be required.

The Event Lead should ideally have experience organising events. If they don’t, then they should have a mentor assigned so they can learn as they go along. The mentor and the main stakeholder do not have to be the same person and often won’t be.

The Event Lead is answerable to the stakeholders - if there is more than one stakeholder, there should be a Lead Stakeholder who can make the final decisions.

Feasibility Stage

The Feasibility Stage comes after the Idea Exploration Stage is signed off by all necessary stakeholders. Now, the event idea is being explored more thoroughly. At this stage, various things are needed before a formal sign-off and progression to the Event Project Stage (i.e. before the Institute/main stakeholder agrees to take on the staff effort, financial risk and opportunities afforded by running the event).

This is a list of information from the Feasibility Stage which is needed for evaluation by stakeholders:

• Goals & objectives

  • What we hope to achieve by running the event (goals) and a set of measurable steps towards achieving our goals (objectives).

• The audience

  • Identifying which groups of people you are planning the event for helps set the goals and objectives in context.

  • Consider your target audience’s career stage, geographic location, gender, ethnicity, job roles, organisation types (e.g. funders, publishers, researchers, administrators).

  • Prioritising the audience might also help with refining goals and keeping planning on target. If the priority changes then this should be reflected in the goals/objectives.

    • Although if this does happen, the Lead Stakeholder should probably be consulted to make sure it’s OK to carry on if you are in the Event Project Stage.

• A date

  • The date needs to fit into the calendar of other events that are likely to affect the target audience.

  • The event should be at a time when accommodation is available. Things to consider here may include:

    • Being outside of University term time
    • Avoiding conflicts with local events (e.g. football matches) as they tend to raise accommodation prices
    • Avoiding clashing with school holidays

• A venue (see an example venue specification)

  • A specification of venue requirements should be put together.

  • Visits should be carried out before hiring a venue.

    • Having an agent local to the event at hand can help a lot when the event is remote to your location.

  • The venue should be checked for availability for the preferred dates.

  • A possible venue should be chosen (or a few choices shortlisted).

• A budget (see a budget example)

  • These should show estimated income lines, and estimated outgoing costs lines.
  • Best case, middle case and worst case cost projections should also be prepared.

• Outputs and outcomes (what’s the difference)

  • These should be linked to the event goals.
  • Some of these will be the same as the objectives of the workshop.
  • They should include the type of resources that will be available after the event. That is, whether a report will be written, videos of the sessions or any other resources will be made available.

Assessment

Once collected, the above set of information will need to be assessed by the stakeholders (or the Lead Stakeholder). The assessment should have one of the following outcomes:

• Proceed - the Event Lead can now arrange the event.
• More work needed - More detail might be needed for some aspects of the event before getting final approval (e.g. re-visit the budget).
• Reject - it might be decided that it is not worth running the event (this should be rare at this stage as most things should be caught at the less expensive Idea Exploration Stage discussions).

Event Project Stage

The idea was accepted and the constraints and practicalities have been signed off. Now we are in the Event Project Stage - it’s time to organise and deliver the event itself.

The Event Lead needs to think about infrastructure, setting a timeline, committees and roles needed in the run-up to the event, the details of the agenda, publicity, how to go about attracting sponsorship, catering, event roles on the day and even how to close down the event.

The following sections go into these elements of the project in more detail and give advice on how they should be developed.

Infrastructure

There are several systems which are useful for organising events which an Event Lead may want to use.

To manage event registration, the Eventbrite system is very useful. There is a fee for paid events (that’s how the system is supported); for free events, there is no fee. You can add custom questions, export registrations to another system, e.g. Google Drive Sheets (or Excel, LibreOffice Calc), to help see how people have answered questions about participation or catering requirements to help with further planning. It has an integrated email system so reminders can be sent to people before they attend, e.g. what they should bring along such as laptops. In terms of payment, there is also an option to allow participants to request invoices, as well as raising invoices at the organisational side so it can add to overheads. This option can be essential for allowing certain groups to attend; however, extra effort is needed to keep track of these and chase these up with reminders.

The Institute has its own website (www.software.ac.uk) which runs on the Drupal content management system. Although information can be placed on the Eventbrite system about the agenda, accommodation options, guidance and other matters relating to the event, it is better to have these on a dedicated website. GitHub, via GitHub Pages, allows the ability to set up a website and may be useful for an adhoc event where access to existing website infrastructure or a content management system is not available.

If your organisation has a list of people who have signed up to hear news or you have permission to contact them after a previous event, then an excellent system to use for promoting to them could be an email marketing system. An example of this is the Mailchimp system which allows the creation of named lists of email addresses and scheduled mailshots amongst other features (e.g. integration with Eventbrite). Sometimes the personal touch may be better and a named person emailing to a list may be more effective. Do make sure to adhere to any applicable data privacy regulations (e.g GDPR in Europe). Also if mailing lists are not being used nor mail merge then remember to BCC multiple recipients so as not to unwittingly share email addresses.

Contacting sponsors individually and having one person emailing them gives better results. Those responsible for the first contact with potential sponsors can be different from those who take over the conversation if sponsors show interest.

Another example of organisational tools are those that help support scheduling; e.g. for talk orders, reviewers or collaboration groups where certain constraint combinations (role, gender, domain or career stage) are used. Doing this completely by hand could be tedious, so Python libraries for handling data, such as Pandas, and linear modelling libraries, such as PuLP - could be helpful to create an initial configuration of people which you can then manually tweak, thus saving a lot of time.

Timeline

It’s important to think about a timeline for the project as soon as it’s confirmed. An outline draft and a more detailed draft at the Idea Exploration Stage and Feasibility Stage would be useful but not essential.

In the Event Project Stage, a detailed timeline will be needed and should be revised and updated as the project develops to reflect changes.

An example timeline is available to give a holistic view of some of the considerations in this area.

It’s preferable to open registration for the event as soon as possible in the Event Project Stage, as this gives more notice and makes it easier for people to commit to attending. In unconference events where the agenda is progressively elaborated people may wait to see a more developed agenda before registering. For events which have a more traditional (e.g. call based) programme, it makes sense for the (near) final agenda to be available when registration opens. In the latter case it would be preferable to make the dates of the event, any calls and when registration is likely to be made available as soon as possible (the start of the Event Project Stage would be a good time) to allow people to save the date.

Budget

It is important to think about the budget very early on and have a budget outline ready in the Feasibility Stage for the perusal of the (main) stakeholder(s). Such an outline should contain estimated income and cost lines, as can be seen in an example budget. In the early stages, you should focus on the budget lines that are absolutely necessary and essential for the event to be delivered, for example:

• estimates for the venue hire and catering
• keynote speaker fees, travel and accommodation estimates (if applicable)
• prizes estimate (e.g. for best paper, best poster, best hack day idea)
• hiring poster boards (if you are hosting a poster session)
• lanyards and badges
• printing programmes (unless you are going paperless)
• expected income from registrations
• expected income from sponsors (which may be an unknown at this stage, unless you already have some sponsors confirmed)

Because at this stage you can only estimate, you should prepare the best case, middle case and worst case cost projections. Try and get actual quotes for as many of the above items as possible, as it will make it easier to plan the budget and will give you more accurate figures. For example, at this stage, you may already have a few quotes from different venues but may be less able to estimate the fees and travel costs for your keynote speakers as you do not know where they will be coming from. In this case, you may decide to put an upper limit on those costs which would also count as your worst case cost projection and will also limit you in your keynote search (e.g. you may decide that you can only afford speakers from a particular geographic area).

You should also estimate a number of optional costs, which you can decide to incur should you attract enough sponsorship, for example:

• conference dinner
• drinks with conference dinner
• conference dinner entertainment
• registration fee waiver for the members of the organising committee
• T-shirts for the members of the organising team and local volunteers (to make them stand out from the rest of the audience)
• conference pack
• better food options
• childcare

As you move further into the Event Project Stage, you will refine and update your budget periodically with the actual costs (e.g. when you sign the contract with the venue or book travel for your speakers) or income (e.g. as registrations start coming in) and make decisions on optional/additional costs.

Finance

Procurement

Procurement needs to be well managed throughout the project. If costs are being met by an institution then the rules and regulations of the institution need to be followed; otherwise, you risk delays or even not being able to pay suppliers leading to a real loss of reputation as well as a lot of aggravation and potential personal liability. Thus, some due diligence here can pay dividends. Have a meeting with your institutional finance people, talk to them about some of the lines in your Budget and get their view on how they should be managed.

Here are some common points that can appear during such conversations:

• All institutions will have a procurement process that you will need to follow. This typically involves determining your requirements for the service or goods, obtaining quotes, choosing and confirming the supplier and creating purchase orders.

• Institutions will also have limits on the amount of financial liability that they will allow you to take. This is effectively the amount that you can “borrow” from them to pay suppliers before you have received the income from the event. If you are running a large event, you will want to discuss this with your finance contacts, as well as considering taking out event insurance - most institutions will have a preferred insurance broker/supplier.

• It is good practice to get multiple quotes before choosing a particular venue and venue related services (e.g. catering).

  • Most institutions will ask for three quotes, though this might be relaxed for smaller value items or specialist items with few suppliers. The level of documentation will increase with the value of the service or goods being procured.
  • In the UK, purchases over £50,000 (including multiple payments for the same service) require a formal invitation to tender process - consult your finance team. Purchases in excess of the EC limit (currently £172,514) require formal advertisement.

• Suppliers need to be registered with an institution before a purchase order (PO) can be created. If they are not already registered as a supplier this should be done as soon as possible, as part of the process of generating a Purchase Order (PO).

  • Sometimes one-off use of a supplier can be done using a different mechanism that does not require registration; check to see if this is an option.

• A Purchase Order (PO) sets out the detail of the goods or services you’re buying from the supplier along with the price and is created once the supplier has been chosen and the details confirmed. This must be done before the date on which the goods/services are supplied, and must be done before an invoice can be paid.

  • Make sure you understand whether the purchase is liable for VAT. Some small suppliers may be VAT exempt, and some activities are zero VAT. Consult your finance team for help if you are unsure.
  • Note that POs are not always used for foreign suppliers - in this case, an agreement or contract is normally used instead.

• Once the goods or services have been delivered, the supplier will send an invoice. This should quote the matching purchase order (PO), with any discrepancy in value explained.

  • Ensure that invoices are processed promptly as many institutions have specific deadlines for payment runs
  • In some cases, e.g. venue hire, you may be asked to pay a deposit in advance. These should be associated with the same PO, to avoid double payment.

There may be other, non-procurement, requirements from an institution relating to the use of suppliers. These can include Data Protection Impact Assessments, e.g. when personal information will be passed to or provided by a supplier and whether suppliers have a statement detailing their approach to the problem of Modern Slavery.

Income

If the financials are being run through an institution, it is also important to think about how income is handled.

Some considerations include:

• How is sponsorship best claimed

  • Will it go into an existing project related code or are there special codes that can be used to hold the money (e.g. ones that might allow greater flexibility)?
  • Does your institution have ethical policies around fundraising and sponsorship that you need to comply with (e.g. requiring due diligence on modern slavery)?

• Does your institution take a cut on any income from events (e.g. for processing payments or running registration services)?

• Will other automated income (e.g. via Eventbrite for registration) be routed correctly? Most institutions will require information about the exact amount of income, the source and a reference to be able to route it properly.

• What is the support for chasing people who paid by a purchase order, who will be able to raise the appropriate invoices?

• Are there any tax implications for income (e.g. needing to charge VAT on non-exempt events - typically non-research events)?

Again having good connections with the financial staff at the institution which is where the budget is held is necessary to make sure the procedures are clear and known.

A final consideration is understanding when your institution’s financial year end falls, and what the implications are for running events. It is not recommended to run an event that takes place across two financial years, and you should ensure that paperwork is done as early as possible if the event is taking place in the last month of the financial year.

Committees

Committees are great ways to shape the event’s programme and get help with logistics while giving people credit for taking part.

We draw examples from the Collaborations Workshop (CW) series, where there are two committees offering different types of help and advice.

• A steering/programme committee

  • Help with the agenda
  • Direction of the workshop
  • Publicity contacts
  • Keynote suggestions
  • Ideas for sponsors
  • Review of contributions or managing reviews of contributions if you have a different set of reviewers (e.g. for mini-workshops in the case of CW).

• An organising committee

  • Focused on logistics (e.g. who will serve the special meals)
  • Identifying roles at the event (e.g. social media event amplifier)

Different types and sizes of workshops will need different committees, roles and responsibilities. For example, the programme committee may be a standalone committee focused on delivering the content of the agenda through identifying programme goals, running calls, making assessments and recruiting keynote speakers.

The Event Lead should decide what type, number, size and complexity of committees are needed for the workshop.

There is useful guidance available for larger conferences and committee roles published by the IEEE. For training-focused workshops, The Carpentries offer some advice. The Research Software Engineers annual conference attracts 300+ attendees and offers information on the place of committees in the conference.

Once the Event Lead decides on the committees, they should recruit members for the roles - they may do this by reaching out to contacts, sponsors who are willing to serve on the committees and local contacts at the venue (for in-person events). They could also ask contacts for further suggestions and even ask the wider community to volunteer and put in place a lightweight system of assessment before they are co-opted.

Once committees are organised, the Event Lead has to think of how many meetings to have and for how long they should be held. Each meeting should have at least a few standing items, key items to resolve and action points from previous meetings, if applicable. By way of example, the CW series has three to four one-hour steering meetings in the last four to five months of the nine-month timeline.

Agenda

The Event Lead should consider the goals, objectives and the duration of the event to decide the agenda and details, such as the type and duration of particular sessions. The agenda and reasoning around a balance of timings and session types should then be discussed with the steering committee (or equivalent) for comment.

The Event Lead may consider the following types of sessions:

• Informing,
• Exploring,
• Creating,
• Learning,
• Networking.

Informing

This includes sessions that inform and inspire:

• Keynotes,
• Short talks,
• Lightning talks,
• Allowing ad hoc presentations (e.g. additional lightning talks).

Exploring

This includes sessions that help people work together to explore solutions or identify problems:

• Collaborative ideas or hack day ideas sessions,
• Discussion sessions and speed blogging,
• Q & A panels.

Creating

This includes building things together:

• Hack days,
• Bring your own data sessions,
• Paper/standards generation.

Teaching

This includes pedagogical sessions:

• Teaching sessions,
• Mini workshops.

Networking

This includes sessions for building relationships between people:

• Around food (e.g. lunch/coffee, conference/workshop dinner)
  • Protecting these times is important to allow networking to take place.
• Other social programmes
  • Guided walks,
  • Visits to museums or other points of interest nearby.
    • At the end of the day or first thing in the morning can be a good time for these to be scheduled.

Publicity

Publicity is key to attract specific audiences to your event and meet your goals and objectives. Publicity happens from the run-up, to publicise and then amplify your event, till after the event, to disseminate outputs and circulate materials from your event to attendees and other potential audiences.

Before the event

Having a publicity plan for an event is vital. The Event Lead will want to reach existing audiences, as well as new audiences or underrepresented groups (e.g. by age, gender, career stage, ethnicity, etc.). The publicity plan should be based on the goals and aims of the event developed as part of the Feasibility Stage.

It is important to investigate dissemination channels and devise a publicity timeline, which references a content plan – the different types of content you will publish throughout the timeline to encourage potential delegates to register for the event. For example, publishing a draft agenda/programme or a call for participation as far in advance as possible will give potential attendees an overview of the event, while keynote speakers will attract different audiences depending on their interests. Sharing experiences from past attendees in the form of blog posts can also encourage people to attend as it might clarify what to expect.

The publicity timeline should fit into the overall event project timeline, and it should include:

• Who to contact (individuals, groups, press offices)

• Contact details (individual email, Twitter, mailing lists)

  • Note : Contact details should be stored securely and only accessible to those who need them

• Publicity schedule (plan the dates for publishing your event and publicity content)

• Content (news items, blog posts, press release, tweet)

• How to measure the success of your campaigns

Once you have your timeline ready, you can start writing or requesting the content to have it ready for publishing. Different tools (such as Google Analytics, Twitter Analytics) can be used to keep track of webpage visits and link clicks. This is useful to assess if your campaign is working and if people are interested in your content.

Photography at the event should also be considered and whether a professional, volunteer or one of the organisers are taking the photos. Attendees should be made aware that photography/filming will be happening, what the images will be used for and that they have the option not to be featured. A conversation should be had before the event about synchronisation between the photographer and those doing social media amplification, taking into consideration any preferences (e.g. via name badge stickers) of whether people approve of or disapprove of their photo being taken or used. As well as recording sessions for reasons of publicity, you may plan to record a short promotional video with different participants talking about the event. This should be planned before the event and potential interviewees contacted in advance. An example of such a video is one produced at CW19 which focuses on Why people like attending.

During the event

A Twitter hashtag for an event or other social media identifier is a useful way to promote the event to the audience outside of those who are attending or to improve the engagement of those in attendance. You would typically arrange staff or volunteers to be in charge of event amplification via social platforms before and during the event. Note that social media depends on audience and demography. Make sure the Twitter hashtag isn’t already being used by a different event or programme.

Amplification stimulates conversation in a space that involves attendees and those who are following along but not attending. If a member of staff or volunteer is in charge of event amplification, they can also pass on some of the outside questions back into the workshop at the appropriate time. It’s recommended to have a list of pre-prepared tweets ready for the social engagement team to send out on the day at specified times. If it’s possible to stream talks, then this can also aid in promoting the event.

Anyone shooting official photographs, whether a member of staff, a volunteer or a professional, should consider any preference around people’s privacy preferences. The person handling the social media would ideally have access to any official photos as soon as possible (e.g. within minutes) for timely use. If you’re producing promotional videos, make sure there is a space designated for the recording to take place and that participants are informed ahead of time of any specific questions.

After the event

Asking for feedback from attendees after the event can be useful for further publicity.

Asking attendees for blog posts to talk about their experiences can offer a fresh perspective of the event. These resources can also become a useful record for future events to help people understand why it might be worthwhile attending your events.

There are various outputs produced at the end of a workshop. For Collaborations Workshops, we usually make available:

• Lightning talks
• Keynote slides
• Hack day entries
• Videos of the talks
• An event report or analysis

Sponsorship

Sponsorship for events has become essential to help make them affordable. We also attract sponsors to events to, for example, pay for a student to attend or support the attendance of speakers or people outside of the usual remit (e.g. legal staff). This, in turn, attracts different audiences to the event and raises the profile of the sponsoring organisations. Sponsors offering free t-shirts, software licences or even hardware can also enrich the experience of attendees.

One has to be careful with sponsorship – there is a delicate balance between in-kind and actual cash. Sometimes in-kind sponsorship can help enrich the programme, but it does not help your budget.

The Event Lead should use the goals and themes of the workshop to identify which companies might be interested in sponsoring your event.

A set of packages (e.g. the traditional Bronze, Silver, Gold, Platinum) should be created before approaching any sponsors. It’s a good idea to leave room for ad hoc sponsorship and be clear about what advantages (e.g. presence on all organiser intro slides) or influence sponsors will have (e.g. a talking slot, a workshop slot, a stand) before and during the event.

As mentioned in the Infrastructure section, members of the event team should get assigned the task of attracting sponsorship. After successful first contact by the event team, the Event Lead can then take forward the conversation with any potential sponsors.

Venue

Securing a suitable venue is one of the most critical aspects of an event and can make the difference between a good and an excellent event. It is also (typically) the most expensive line in your Budget.

You can have a look in more detail at various venue requirements you should consider in the venue requirements template document and adjust it based on your own capacity and programme needs. Remember that venues may not meet all of your ideal requirements and you will have to prioritise what is a must, what would really help and what is optional for your particular event. Here we outline various aspects of venue requirements which you should consider and prioritise for an event.

• Capacity

  • The main plenary room’s capacity should be sufficient to comfortably fit the number of delegates, and there should be enough rooms and spaces to support different event requirements

• Areas that support additional event requirements

  • Registration, main plenary room, breakout rooms, communal/break space, poster/sponsor exhibition, storage, quiet space, workspace, video and photography, prayer room, etc.

• Location

  • Ideally located in a place with some wow factor and easily accessible by public transport (ideally with some nearby parking and cycle parking too)

• Infrastructure

  • WiFi, A/V, video streaming/recording, power points and portable microphones

• Catering at the main event

  • Catering for people with different dietary needs and food allergies, clear labelling of food

• Facilities

  • Storage, toilets, fridge for medication/milk, water points, printing, childcare on-site (or nearby nursery), proximity to prayer rooms (if none available on site)

• Accessibility

  • Accessibility within the venue (location of disabled toilets, accessibility of various rooms, signposting for ease of orientation)

• Environment sustainability considerations

  • The use of plastic vs sustainable cutlery and crockery for serving food (guests can be encouraged to label and reuse their cups throughout the day if disposable), consider serving only vegetarian food or having guests opt-in for meat options, what happens to the leftover food?
  • Minimise printed materials e.g. make slides available digitally

• Conference dinner hosting

  • The possibility of hosting the conference dinner at the venue

Catering

The budget projection at the Feasibility Stage should already reflect catering prices, including any added expenditure that might incur from late registrations. The budget template shows an example of a catering order.

The Event Lead or designate should ask for the delegates’ dietary requirements in advance, preferably at registration time. These requirements should inform the catering order for the workshop.

The Event Lead (or designate) should find out from the caterers the latest date to submit an order. It might be worth ordering 5%-10% extra meals to allow for late registrations between ordering the food and the event.

It’s tempting to order the cheapest things on the menu to meet the budget, but having variety, quality, and reasonable portions (especially at a multi-day event) can help keep the participants fuelled for learning and interaction. If you are charging your audience to attend the event, then it is best to give careful consideration to catering.

It can be time-consuming to arrange catering so this might be best assigned to another member of staff during the task allocation process.

Accommodation

When selecting a venue ask about nearby accommodation options. Consider proximity to the venue and transport links, price range, disabled access and if the venue recommends using a particular provider. Once a final venue has been decided then accommodation needs to be managed more proactively.

Various approaches towards accommodation could be taken:

• Direct people to find their own

  • This decreases the burden on the organisers but may not create the welcoming and caring impression desired for the event.

• List nearby or recommended accommodation providers on the event page

  • It’s important to be clear about whether these are a curated list based on recommendations or just local options. Inform participants if any special discounts have been arranged.

• Arrange accommodation for all or some of the participants

  • This might be the best way to support staff or invited participants.
  • This may be done to arrange a larger discount in a nearby venue which can then be charged appropriately to participants during registration. There are benefits of attendees at the event in staying at the same hotel, for example, it can make organising social activities and coordinating transport easier.

If accommodation is being arranged for some or all of the participants then investigating the different hotel policies around cancellation and final numbers should be done as soon as possible. All things being equal, hotels which have policies which enable cancellation or transfer of the booking closer to the event date are preferable as this decreases the period of time when a cancellation would lead to financial loss.

Event Roles

Certain roles (see examples roles) are needed before, during and after the event. These roles can vary depending on the types of sessions and activities at the event and some of the roles naturally go together. Role allocations should be done carefully as some combinations of roles might be incompatible or require local knowledge or skills, so they would only be suitable for certain people.

A duties roster

For medium-sized or large events, it is essential to put together a duties roster (see example roster) before the event, so people with different roles know what they need to be doing at specific times. Scheduling duties decreases problems with communication during the event, as every team member clearly knows what they’ve agreed to do at specific times throughout the event. Duties usually forgotten when creating the schedule are registration and reception.

The Event Lead may already have an idea of who should be included in the roster. For example, the Collaborations Workshop is the premier event of the Institute, so all staff are expected to attend and possibly have a role. Where this is not the case, doing a call out to the organisation or even asking a local contact for volunteers may be the right way to go. There is a long tradition of PhD students helping at events, though you should consider rewarding them for their efforts at the end of the workshop.

Event Safety

The safety and comfort of participants at the event are paramount. To make sure the event will be safe and comfortable, information and processes need to be available and in place.

Safety information

A minimum of information should be available and at hand for organisers and participants, including:

• Venue contacts

  • Medical emergency number
  • Site security number
  • Fire warden number or help number

• City contacts

  • Medical emergency number (e.g. 999 in the UK)
  • Security emergency number (e.g. for the Police)
  • Address of the nearest accident and emergency equipped hospital
  • Address of medical practitioners who offer walk-in services
  • Address of the nearest pharmacy

Code of Conduct

As well as medical and security matters, governed by the institutional and legal authority of the venue, a Code of Conduct will mandate standards of behaviour and interaction expected from organisers and participants. These are becoming increasingly common at events and show a commitment to running a safe event. In some cases, they need to be in place before potential sponsors agree to sponsor the event.

The Code of Conduct should come as a set of documents:

• The Code of Conduct itself (e.g the CW19 Code of Conduct)
• How to report a violation (e.g. the CW19 - Attendee Procedure For Reporting Harassment)
• How the Code of Conduct committee will handle harrassment reports (e.g. the CW19 - Staff Procedure For Handling Reported Harassment)

Risk Management

We define risks as situations that can negatively impact an event. In this guide, we refer to risk management as the process of identifying potential issues, assessing their likelihood and impact, and then preparing a mitigation plan to manage them during an event.

Risk management is an ongoing activity throughout the organisation of an event. Potential issues, their likelihood and impact, and their mitigation plans will change as the planning of the event moves forward.

Note - What we describe here is a tailored approach to risk management. In risk assessment theory, risk proximity is also a factor (that is, how close you are in time to a potential issue). We have not included this as the Event Organisation Guide is focused on experiential knowledge and we have not used that approach in our events at the time of writing. However, if you wish to include risk proximity, then you’d have a three-dimensional risk assessment matrix. The approach included here is an example, and the Event Lead should tailor it according to a specific event. Bear in mind that the more complex, expensive and novel an event is, the more effort needs to be devoted to risk management. You can read more about risk management at the Prince2 Wiki.

Risk Identification

The first stage of risk management is identifying risks. Note that as risk management is an ongoing activity, new threats will come up as the event organisation proceeds, existing potential issues will be re-phrased, and some others will be deleted. Risk management will take place until the Closing Down the Event Project Stage.

Here are some examples of risks identified when organising an event such as the Collaborations Workshop or CarpentryConnect.

• The main stakeholder

  • Event goals not aligned with the main stakeholder goals
  • Incidents during the event can damage the main stakeholder’s reputation

• Sponsors

  • Not enough sponsorship
  • Offers made to sponsors that cannot be fulfilled
  • Sponsor’s logo is not displayed at all at the event (or not displayed appropriately), spelling and branding mistakes occur

• Keynote speakers

  • Delay in confirmation or dropping out
  • Speaker goes off track in their talk or uses offensive language

• Attendees

  • Not enough attendees or many dropping out right before the event
  • Too many attendees

• Session chairs

  • Not enough session chairs
  • Ineffective chairing of the sessions
  • Do not turn up on the day

• Organising committee members

  • Not kept up to date with the progress of event planning
  • Not sure how to contribute or how to share information and coordinate between different organising committee teams
  • Not completing tasks assigned to them
  • Do not know where help is needed and what their duties are at the event
  • Feel unappreciated or excluded

• Volunteers/helpers at the event

  • Not enough volunteers
  • Volunteers not fulfilling the tasks assigned to them
  • Feel unappreciated or excluded
  • Do not turn up on the day

• Budget

  • Insufficient budget (e.g. registration fees not set at the appropriate level)
  • Unexpected expenses (e.g. extra catering, damage to the venue)
  • Overspending

• Event agenda

  • Not designing an effective agenda to meet the event’s goals and keep people’s interest and focus
  • Not fulfilling specific plans (e.g. social activities, good venue for conference dinner, etc.)
  • Unexpected bad weather so cannot do activities outdoors

• Venue

  • Venue cancels before the event
  • Problems at the venue (accessibility, AV, fire alarm etc.)

• Catering

  • Catering cancels before the event
  • Lack of a particular food of preference, wrong food delivered, food not adequately labelled for allergens and dietary requirements

• Communication

  • Participants confused about the agenda or not getting information on the right channels and at the appropriate time/frequency
  • During the event - signage not properly placed, people do not know who to contact to address certain queries or concerns
  • Participants do not receive information about what to expect after the event

• Policies, guidelines and documentation

  • Lack of information or information dispersed at various locations
  • Lack of clearly stated policies and guidelines (e.g. Code of Conduct policy and breach reporting and handling procedures, privacy policy, contribution guide)

See the example risk management template for more information on how to elaborate on identified risks, e.g. using the if-then method for writing up risks can help to formulate and infer mitigation plans.

Risk Assessment

At the time of evaluation, the risks identified should be ranked by their likelihood and impact. The likelihood refers to the probability of risks actually happening, while impact is the consequences if a potential risk happens.

The figure below illustrates the conceptual relationship between likelihood, impact and risk.

Once the risks are assessed based on their likelihood and impact score, they can be visualised on a risk matrix.

The following is an example risk matrix from the Wikimedia Commons. The image is available under the CC-BY-SA 4.0 license).

The event team can then decide which part of the matrix is above or below the project risk threshold. For experienced teams, the risk threshold can be set high. The threshold may need to be lowered if the team is new or lacks experience in running a particular type of event. Mitigation plans will need to be formulated for risks above the threshold.

See the risk management template for an example that includes a risk matrix.

Risk Mitigation

Risk management occurs regularly during the Event Project Stage of the event. For the risks above the risk threshold, mitigation plans need to be put in place.

Common mitigations strategies fall into the following categories:

• Avoidance

  • The risk is avoided entirely by taking a different course of action

• Acceptance

  • The risk is accepted and contingency plans are made in case it happens

• Transference

  • We hand the risk over to someone else, e.g. via insurance

• Limitation

  • Plans are in place to limit the likelihood or impacts of the risk identified

The example risk template contains risk mitigation plans for risks above the risk threshold in the example event.

Outputs and Outcomes Plan

Outputs

It’s important to plan what you will make available after the event. For instance, for a Collaborations Workshop type of event these normally include:

• Presentations

  • From keynotes (slides, videos)
  • From lightning talks (slides, videos)
  • From mini-workshops (slides)

• Speed blog posts

  • From discussion sessions

• Collaborative ideas

  • Making the final ideas available

• Hack day

  • Information about entries and winners
  • Team/Project name
  • Presentations
  • Links to the software/output

Outcomes

Outcomes tend to match objectives but the Event Lead should decide if a report or thematic analysis of the workshop outputs might help highlight any change of outlook due to the workshop. Feedback results can also be part of this. Longer term analysis (e.g. surveys after the event may also be useful to see if the workshop has, for instance, changed practice or led to further collaborations). See the Ten simple rules for measuring the impact of workshops for more ideas and guidance.

Closing Down the Event Project Stage

Closing down the Event Project Stage requires effort and planning - it’s crucial for organisational reputation and following up on commitments, promises and making resources available.

At the end of the event

Attendees should be made aware of any activities which need action from them; for example, the Event Lead will be gathering feedback or following up with them about speed blogs.

Also, if you’re aware of other events or a follow-on event, making attendees aware of these will help meet the wider goals and objectives of your organisation.

Sponsors and volunteers should be thanked again at the end of the event.

After the event

Soon after the event, there are some tasks that can be done straight away and some other which might take a bit more time to complete.

If the Event Lead collects feedback, then forms should be ready to send the next working day after the event - the danger of leaving this for longer than a week is that people will start to forget what happened as they get back to their normal routines. Ideally, the email asking for feedback with a link to the feedback form will be ready before the start of the event.

For larger events, the Event Lead may request blog posts about people’s experiences at the event. This request should also be made in the week following the event.

A full report of the event can be time-consuming to produce, so it’s worth thinking about which type of report you want; e.g. a paper on major themes, an event report that has a ‘being there’ style, or a summary of resources produced and available from the event. From experience, writing the report can take up to four months after an event like the Collaborations Workshops, as the content is dependent on reviewing what happened and other resources made available.

If resources (such as presentations, video recordings, speed blog posts and outputs from a hack day) will be made available, then these need to be commissioned or collected and prepared. You also have to decide when to publicise the materials (e.g. as they are made available or when they are all ready) - this should be part of your publicity plan.

It’s important to take the time to make sure that the financial side of the event is completed. Ensure that all income from registrations is recorded, collected by the platform you are using to handle this, and paid out / transferred to the (e.g. institutional) account you’re using for event finances. Invoice any remaining sponsors and check that they have paid. Make sure that all invoices from suppliers are received, checked and paid. Check that there are no outstanding items or liabilities to be discharged (e.g. refunds). Once this has been don,e you can update your event accounts to provide a final statement for the event, which indicates any surplus or deficit. The final statement can be compared against the initial budget of the event and help plan subsequent events.

When are you done?

When all of the outputs you planned to produce are available, you can consider the output side of the event done.

For outcomes, the Event Lead may want to track longer-term changes – for example, a longer-term survey (~after six months or a year) for attendees might be useful to assess the impact of your event.

Once you have produced your final accounts for the event, you can consider the financial aspects complete.

If this is all you planned for, then the event is done.

It is possible that page hits, resource usage, and citation of the report might feed into tracking broader organisational impact and those in charge of this should be made aware of these resources.

In Practice

What follows are in-depth write-ups of using the Event Organisation Guide in practice. We also refer to them as In Practice write-ups for short.

Collaborations Workshop 2020 (CW20) - Online

In Practice authors: Rachael Ainsworth (ORCID) and Shoaib Sufi (ORCID)

The Collaborations Workshop (CW) is the annual flagship event of the Software Sustainability Institute. It brings together researchers, developers, innovators, managers, funders, publishers, leaders, and educators to explore and share best practice.

On 5 March - just over three weeks before Collaborations Workshop 2020 (CW20) was due to start in Belfast, Northern Ireland - everything was in place. The programme had been finalised, keynote speakers announced, catering booked and accommodation assigned. There had been some issues - a group from Northern Italy had cancelled because they had been quarantined due to an outbreak of a novel coronavirus. At this point the number of reported cases in the UK was small, but things were moving rapidly. The clear message was that we couldn’t run the event as planned.

CW is a highly interactive event with around 80-100 participants, designed to facilitate meeting as many new people as possible. With a global pandemic forcing us to stay at home, could we successfully recreate the experience online and with only three weeks to prepare? Given the amount of work that had already been put into organising the event, the decision was made on 10 March to move the event online and stick with the original dates: 31 March - 2 April.

Here is how we did it. Note, this Event Organisation Guide In Practice write up takes place just before our decision to go online and covers the steps in the Event Project Stage of the main Event Organisation Guide. We do not go into details about the face to face CW20 event that was planned. This indepth write up runs until the resources after the workshop were made available.

For a shorter case study that covers the move in summary please refer to the blog post CW20: how to move an event online in three weeks.

CW20 Infrastructure

We had already organised registration for the in-person event through Eventbrite. We emailed registrants through Eventbrite to notify them of the decisions that we were making and thereby keep them informed. Because of the unconference nature of the event, we even asked for participant input to our contingency investigation; surveying them to see if they preferred the event to stay as an in person event, move to online or be cancelled.

As we worked to reformat the event and move it online, we gave registrants time to cancel and collect full refunds. We also reduced the cost of the event (from £250 to £50) and issued partial refunds through Eventbrite to participants who wanted to stay and participate in the virtual event. We took the partial refund route in order to maintain continuity with our registrants, but another option could have been to outright cancel the Eventbrite event, issue full refunds and start from scratch. The option to outright cancel the Eventbrite event was worth considering, as an Eventbrite fee would be imposed on registrants if they requested refunds for no longer being able to attend as per the refund policy. (Although we note that Eventbrite eventually added a COVID-19 option as a reason for cancelling registration and requesting a refund, which did not impose a fee on the registrant or organiser.) We also provided guidance and documentation on reclaiming costs to help participants who had already booked their travel to Belfast.

For the in-person event, we had planned to use additional tools such as Slack, Slido and uCONFLY, however for the online event we felt that it was important to minimise the number of platforms that participants had to register with and keep track of. We narrowed our needs down to platforms for video conferencing, a chat system for sharing information and community engagement, and a tool for collaborative note-taking to keep everyone synchronised.

Platforms

Video conferencing

Zoom was very quickly identified as the video conferencing platform that we would use because we had the most experience with it and it had the functionality we needed (scaling and stability with many video-based participants, breakout rooms and easy recording of the event). We initially planned to use a single Zoom meeting room to contain the entire event (with breakout rooms used for the discussion, collaborative ideas, and mini-workshop sessions). However, workshop facilitators also wanted to make use of the breakout room feature for their sessions, which wouldn’t be possible if they were already in breakout rooms. We therefore obtained five Zoom Host accounts in total to accommodate this: one for the main/plenary room, plus four for the mini-workshops which would take place in parallel.

Chat system

Although Zoom has chat functionality, it can be distracting for hosts/speakers and difficult to manage when there are many participants using it. We decided to disable the Zoom chat (except for the case where participants could communicate directly with the meeting host) and direct engagement and communication to the CW20 Slack workspace, as this would allow participants to connect even when the Zoom room is closed. A communication channel that persists outside the meeting is helpful for maintaining access to resources shared, referencing discussions and facilitating community engagement.

We created practical channels for event organisation and communication, such as a #help-desk channel for participants to direct their technical issues to, and private backchannels for the organisers, helpers, and Code of Conduct committee. Participants were able to create channels for their discussion groups and Hack Day teams. Slack also allowed for a more human and informal engagement through the creation of channels such as #pets-at-cw20, #kids-at-cw20 and #outside-my-window-cw20 for sharing photos of our pets, children and working from home environment during lockdown.

Collaborative documenting

The final resource that we needed to decide on was what platform to use for collaborative note-taking - a method for participant interaction and engagement that we learned from communities such as Mozilla Open Leaders, rOpenSci and NL-RSE Meetups. Google Docs was the obvious choice for CW20, as we generate a lot of Google Docs through uCONFLY for the various sessions, and it also has the lowest barrier to accessibility (no need to log in or register and you do not need to know specific syntax or markdown language such as with HackMD). We created a note-taking document for each day which contained the full agenda and all relevant event information, a space for roll call, instructions and links (links to Zoom rooms for parallel sessions, discussion group and collaborative ideas documents, etc.) as can be seen in this example collaborative document.

There were spaces created within the document for people to take notes during the talks and to ask any questions; all of which could be preserved and referred back to in the future. Crucially, general guidance was included at the top of the document reminding participants of the Code of Conduct (and reporting procedure), that the event was being recorded (so to mute any audio/video as necessary), security precautions (warned not to share any of the links publicly to avoid the meeting getting sabotaged by “Zoom-bombers”), and all the pathways to engagement (notes documents, Slack workspace, etc.). We also provided space for feedback at the end of the document on what worked well during the workshop and what didn’t so that we could adapt for the next day.

One limitation of using Google Docs is that only 100 people can edit and comment at the same time - which we experienced at the start of the event. Although it requires some knowledge of markdown syntax, HackMD can support 200+ concurrent people accessing a document.

Normally, we require CW participants to register on the SSI’s unconference resource management system uCONFLY, which allows them to request instances of Google Docs for their group sessions and vote for the best collaborative ideas. It also allows all of the outputs from CW to be centralised. For CW20, we used uCONFLY purely as an admin tool by creating the templates for the discussion, collaborative ideas and mini-workshop sessions, and generated all of the needed instances ahead of time ourselves. We then collated them via Google Sheets (see this example spreadsheet) which we linked to in the day’s collaborative note-taking document/agenda.

Resources

You can find the full collection of CW20 infrastructure and documentation resources on Figshare.

Resources included that were used behind-the-scenes:

• An index of links to documentation and event resources for organisers to easily navigate on the day.
• A duties roster with lists and descriptions of the extra roles we needed for our online event.
• A spreadsheet for the mini-workshop and demo sessions assignments to demonstrate how we organised parallel sessions.
• Zoom instructions for events with parallel sessions to make sure all account hosts are on the same page.
• A checklist to remind the event chair what tasks need to be done before and at the start of the event (such as a reminder to send the connection details, assign co-hosts, record the event and take a group photo).

Resources included that were participant-facing:

• An example collaborative notes document (with agenda) to guide participants through the event, provide another pathway to engagement and compile questions, notes and outputs from the event.
• The spreadsheets which we used to facilitate our unconference sessions (discussion groups, collaborative ideas and Hack Day).

CW20 Timeline

We made the decision three weeks before CW20 to move the event online. We had less than one month to contact all of our stakeholders (venue, accommodation, sponsors, speakers, workshop facilitators and of course participants), inform them of the decision and renegotiate everything that had been previously agreed. As days passed and the implications around the global pandemic became more clear, stakeholders were very understanding and accommodating. Below is a breakdown of what we did.

Event Project Stage

Month -1 (March 2020)

Agenda

• We reorganised agenda for online format by:
  • Removing in-person social programme (walk in Belfast Botanical Gardens, visit to Ulster Museum, Drinks Reception, Conference Dinner)
  • Adding breaks and buffer time
  • Adjusting timings of sessions and rearranging order
• Confirmed lightning talk schedule
• Made the final agenda available

Communication

• We closed registration and contacted registrants through Eventbrite as soon as the decision was made that the event would be moving online to notify them, and included information on:
  • The full refund policy if they were no longer able to attend the event (the deadline for a full refund was 7 days before the event start date)
  • The reduced cost of tickets and how they would be refunded if they still wanted to attend (partial refunds for the difference would be issued during the week before the event, after the deadline for full refunds passed)
  • Guidance on how to claim back travel and accommodation costs
• We had discussions with the venue and accommodation about postponing the in-person arrangements to a future date
  • We were able to rearrange everything (venue, catering, accommodation) except for one of the hotels which was unable to refund/reschedule our reservation under the circumstances
• Followed up with sponsors and offered to renegotiate packages
• Followed up with speakers and workshop facilitators, confirming any changes to their sessions, and provided office hours to test their setup and answer any questions
• Contacted the event delivery team with new duties roster, held pre-event meetings to test the infrastructure and make sure everyone was comfortable with their role
• In the days before the event, we sent emails to registrants through Eventbrite reminding them to review the Code of Conduct, instructions to install and test the Zoom software, and information about how to prepare for certain sessions
  • We also stressed that CW20 would be as flexible on the day as possible, as many people were adjusting to a new way of working, caring responsibilities during the day, unstable internet, and to the associated emotional pressures
  • We provided guidance for everyone to be kind, understanding and flexible
• Links to the relevant Zoom room and note-taking document were sent via email through Eventbrite and Slack at the start of each day

Resources

• We set up a uCONFLY instance for management of discussion group reports, mini-workshop and demo session notes, collaborative ideas and Hack Day ideas which also involved:
  • Creating Google Doc templates for each of these document types
  • Generating a number of each document type for dissemination during CW20 so that participants would not have to login and use yet another platform
• Built the infrastructure needed to synchronise participants and run the unconference sessions such as:
  • Collaborative notes documents with agenda and relevant links
  • Spreadsheets to disseminate the uCONFLY generated documents
• Wrote documentation for the delivery team including:
  • Zoom instructions
  • Duties roster
  • Links index
  • Start of day tasks for Event Lead/Chair
  • Mini-workshop and demo session assignments
• Made organiser slide deck

Event run up actions

• Closed registration immediately upon decision to move event online
• Issued full refunds to registrants who were no longer able to attend and informed us at least 7 days before the event start date
• Issued partial refunds for the difference between the original priced and the new reduced price to registrants who still wanted to attend
• Planned break-out groups distribution (sign up sheet for discussion groups session, randomly assigned for collaborative ideas session)
• Planned for video recording infrastructure (easy recording of sessions via Zoom)
• Had a Code of Conduct committee pre-event meeting
• Constructed feedback form draft

Month 0

Ran event!

• Overall, CW20 ran smoothly according to plan
• Places where we feel we can improve include:
  • A smoother running of the Hack Day: assigning co-host privileges to Hack Day participants to allow them to move around breakout rooms was cumbersome (although we note that as of Zoom version 5.3.1, there is now the option to allow participants to choose their breakout rooms) and participants found it difficult to break into sub-teams
  • Mingling, socialising and “hallway conversations” in online workshops

Month 1 (April 2020)

Agenda

• Updated the agenda with links to resources

Feedback

• Emailed participants with a feedback form within a week after the event
• Sent a reminder email a few days later
• Held a feedback prize draw to thank them for taking the time

Publicity

• Published a blog post with highlights from the event
• Published a participant’s experience of CW20

Resources

• Updated language of event website to show that it was in the past
• Created a Figshare portal to share CW20 outputs
• Chased speakers for any un-captured resources
• Collected Zoom recordings from the Hosts/Co-Hosts of each session
• Prepared videos for editing by
  • Organising files
  • Updating processing instructions for Events Team
• Followed up with speed-blog authors and connected them with the Communications Team for publication to the SSI blog

Month 2 (May 2020)

Publicity

• Published a reflections blog post about how we moved the event online in three weeks

Resources

• 9/12 discussion groups’ speed blogs published

Month 3 (June 2020)

Publicity

• Announced availability of Infrastructure resources

Resources

• 10/12 discussion groups’ speed blogs published
• Infrastructure resources sanitised and shared

Month 4 (July 2020)

Publicity

• Announced availability of session recordings

Resources

• 100% of videos processed and uploaded

Month 5 (August 2020)

• 11/12 discussion groups’ speed blogs published - the last one may be published in the future if the group members decide to

Month 6 (September 2020)

• Event Organisation Guide In Practice in-depth write up completed and reviewed

CW20 Budget

We were extremely lucky that there were no additional costs for moving the event online - the University of Manchester (UoM) had already provided staff with Education licensed Zoom accounts which allowed for unlimited meeting duration for up to 300 people. There were enough UoM staff members on the CW20 delivery team that we had enough Zoom Host accounts for the main plenary room plus four additional rooms in parallel for the Mini-workshop and demo sessions.

We did reduce the cost of registration from £250 to £50 as there would be no catering or physical venue involved in an online event. The reasons we didn’t make the event free included the staff time that had gone into organising the event, to prevent a large number of no-shows on the day and because there were prizes involved. This caused a significant loss of income but the burden of any costs lost due to the pandemic should not be placed on the registrants.

Costs for things that we had already paid for that would carry over into the virtual event:

• Prizes for Collaborative Ideas and Hack Day

Costs that were able to be deferred to a future date:

• Venue hire
• Catering (including a drinks reception and conference dinner)
• Audio/Visual
• One (out of two) of the hotel block bookings for Institute staff and Fellows

Costs that were lost due to cancelling the in-person event:

• One (out of two) of the hotel block bookings for Institute staff and Fellows
• Reimbursing travel costs of Institute staff and Fellows

CW20 Committees

Steering Committee

The CW20 Steering Committee comprised of representatives from Belfast, the workshop sponsors and the Institute. Members helped to guide the agenda and direction of the workshop, provided advice for running the event and contributed to many of the workshop sessions. Before publicly announcing that CW20 would move online, we notified the Steering Committee of our contingency investigation and decision to replace the in-person event with a virtual one.

Organising Committee

The CW20 Organising Committee comprised of representatives from Belfast and the Institute. Members had more regular communication, with a focus on logistics and identifying and being responsible for roles/duties needed to run the event. Once the decision was made to cancel the in-person event in Belfast and move CW20 online, the committee members from the Institute worked to

• Create and test the infrastructure for running a virtual event
• Reformat the agenda for the online setting
• Adapt the publicity
• Help run the event on the day

Code of Conduct Committee

The CW20 Code of Conduct (CoC) Committee comprised of three members who were Institute staff and one external. There was a mix of gender, race and career stage within the Committee to provide a range of options for participants to choose who they felt most comfortable reporting to.

Before the event, we held a meeting for CoC Committee members to read through and discuss

• The Code of Conduct
• The Procedure for Reporting Harassment
• The Procedure For Handling Reported Harassment

During the event, we had a private Slack channel in the CW20 Slack workspace for CoC Committee members in case any incidents arose. No misconduct was reported from CW20.

CW20 Agenda

For in-person CWs, the days are long to maximise the opportunities for networking, collaboration, discussions, and solving problems across the research software landscape. The agenda that we had spent months organising was packed full of talks, group sessions, social activities and mini-workshops - we originally had 18 mini-workshops scheduled to take place across five sessions.

If we had organised CW20 as an online event from the beginning, there are many things we would have done differently. However, we found ourselves in the position of having to transition from an in-person event to an online one at short notice, so we still wanted to incorporate as much of the original programme as possible. We contacted speakers and workshop facilitators to ask if they were still able to deliver in the online setting. In the end, two of the workshops dropped out, and we were therefore able to shift the others around so that we had 16 workshops across four sessions (with four workshops in parallel for each session).

Because we were now unable to carry out the planned social programme (activities such as visiting the Ulster Museum, a walk around the Belfast Botanic Gardens and of course the conference dinner), we were able to shorten the days and add in further breaks and changeover/buffer time.

We did not organise online social activities during the breaks because the days were still quite long and we thought people would need time away from their screens, but this was ultimately something that participants missed the most from the in-person event.

Breaks were specifically scheduled to allow time for the creation of breakout rooms for the next session.

Sessions

Icebreakers (Networking)

We started each day with an icebreaker to get participants used to the infrastructure and to have a chance to chat in small groups. We asked a couple of yes or no questions for participants to respond to using Zoom’s non-verbal feedback feature, and then randomly assigned them into breakout rooms of 3-4 people to discuss the following prompts which we thought would be both fun and useful:

• What are you reading, watching and/or listening to at the moment (and why)?
• What advice/resources/tools/guides do you find useful for working remotely (and why)?

We then asked participants to write down their answers in the note-taking document during a few minutes of “silent documenting”, which had the added benefit of empowering people to directly contribute to the shared notes (example icebreaker document).

Keynote Talks (Informing)

Keynote speakers were contacted prior to the event to confirm the time, length and format of their talk.

The Chair of the session introduced the speaker and reminded the participants that they could take notes and ask questions in the collaborative notes. A link to the slides was also shared here.

The speaker shared their presentation slides via Screen Share in Zoom, and the Host set Spotlight Video for the speaker so that they would be visible to all participants and in the recording.

During the Q&A, the Chair then relayed some of the questions from the notes to the speaker, and the remainder of the questions were answered in-line within the note-taking document after the session.

Lightning Talks (Informing)

All presenters submitted via Dropbox their slides for lightning talks. We compiled all the slides into a single slide show to maximise efficiency during the session, and the Chair shared them via Screen Share in Zoom. The lightning talk speakers then unmuted themselves to give their talk, and the Chair kept them to time. A lesson learned here is to have a timer (e.g. Cuckoo) visible for speakers to know how much time they have left. This can be done by sharing a Desktop view with both slides and a timer visible, instead of only sharing the presentation window. Again, the Host sets Spotlight Video for the speaker so that they are be visible to all participants and in the recording.

Discussion Groups (Exploring)

The Discussion session was conducted in breakout rooms. We were worried that the nascent process of participants determining which discussion topics are chosen and which rooms the groups meet in could be difficult to manage online, but it went much smoother than anticipated. Participants suggested and signed up to the topics that they wanted to discuss in a spreadsheet (example spreadsheet). The topics that people signed up to were then assigned a Group ID - in this case using the letters of the alphabet. The Zoom Host then created breakout rooms named Group A, Group B, etc. and assigned the people who signed up to those groups to the associated breakout rooms during the lunch break. The groups were pointed to their discussion group reporting document via the “Groups” tab in the spreadsheet, in which they worked together on their speed blog.

Collaborative Ideas (Exploring)

The Collaborative Ideas session was conducted in a similar way to the Discussion session except that the group assignments for the breakout rooms were randomly generated. This time the Group IDs were kept as numerics, so the group that was assigned Breakout Room 1 retrieved their Collaborative Ideas document for Idea Group 1 in the associated spreadsheet (example spreadsheet), and so on. Normally, voting for the best Collaborative Ideas takes place on uCONFLY, but to streamline things we added a “Voting” tab to the spreadsheet and asked participants to simply add their names to the ideas that they wanted to vote for and tallied these up at the end.

You can read more details about how we ran the Collaborative Ideas session in the Collaborations Workshop section of the CSCCE’s guide to Using virtual events to facilitate community building: event formats.

Mini-workshops and Demos (Teaching)

We had 16 Mini-workshops and Demos across four sessions. We used a separate Zoom account for each of the four parallel workshops in each session, with an assigned Host (and Co-host for backup) for handling the logistics of the workshop (managing Zoom, recording the session, pointing to the relevant notes document) and helping the workshop facilitator with any technical issues. We pointed people to the different Zoom room links in the day’s collaborative notes and in the Slack channel, and someone stayed in the main room to help people get to where they wanted to go. A lesson learned is to guide workshop facilitators towards including interaction from participants for at least half of the time, and not just having one long presentation.

CW20 Publicity

Publicity and clear communication were key for transitioning CW20 from an in-person to a virtual event within three weeks.

Before the event

We published a news item announcing that CW20 would move online. We asked our community to please bear with us while we updated the Eventbrite page, the CW20 website and modified the format of the workshop to make the experience as useful and enjoyable to our participants as possible. We included information on what participants could expect with regards to registration and general advice to participants on reclaiming travel costs.

We updated the Eventbrite page description to include our response to the situation around COVID-19 and that the event would move online, edited the location to Zoom, and changed the registration ticket options and prices to the new reduced rates.

We edited the CW20 website to:

• Update the workshop description and image to reflect that it would now take place online.
• Add a Response to Coronavirus (COVID-19) section notifying the community of the decisions made around the event.
• Update the agenda with the reformatted programme.
• Update the venue information with instructions on installing and using Zoom.
• Update the Code of Conduct to include additional online interactions and reporting.

During the event

We used the Twitter hashtag #CollabW20 to promote the event on social media, following the convention of previous CW events. We assigned a staff member as Twitter Monitor to tweet throughout and amplify the event.

After the event

After the event, we solicited feedback from participants through a Google Form, motivated by a random prize draw for those who submitted it. Overall, we received very positive feedback regarding the general structure, organisation and communication of the workshop, and how useful and enjoyable it was.

We published several blog posts after the event, including:

• Highlights from Collaborations Workshop 2020: a short reflective piece
• SSI Collaborations Workshop 2020: Remote unconference experience and notes: a participant’s experience of CW20
• CW20: how to move an event online in three weeks: a longer reflective piece
• The speed blogs from the Discussion session

We made the keynote presentation slides, lightning talk presentation slides, and infrastructure resources available through a CW20 conference portal on Figshare and announced their availability through a news item after the event. We also made the CW20 session recordings available via the Institute’s Youtube channel and announced their availability through a news item.

Finally, we contributed a recipe for how to run the virtual Collaborative Ideas session to the Center for Scientific Collaboration and Community Engagement’s (CSCCE) guide to Using virtual events to facilitate community building: event formats, which has been viewed 6,000+ times and downloaded 5,000+ times.

CW20 Sponsorship

As soon as we made the decision to move online, we contacted the event’s sponsors to notify them and offer to renegotiate any previously agreed packages. We were extremely grateful that our sponsors Microsoft, F1000 Research, Figshare, eLife and Overleaf pledged their full support of the event moving online and also engaged with the programme through lightning talks, mini-workshops and Hack Day judging.

CW20 Venue and Catering

After a discussion on 9 March 2020 with the local organisers at Queen’s University Belfast, we agreed to postpone CW in Belfast to a future date, with no charges for the change to the room booking dates and catering. As no catering was needed for the reorganised online event, we were able to reduce the cost of attending.

Further Government restrictions and lockdown measures throughout March would have prevented the event taking place in person if we had decided to go ahead with an in-person event.

We decided to use Zoom for the online venue, which you can read more about in the CW20 Infrastructure section.

CW20 Event Roles

It was clear that additional roles and duties were needed for an online event relative to an in-person event.

In addition to the following roles we typically have for an in-person CW event:

• Event Chair/Facilitator (filled in this case by the Event Lead)
• A Code of Conduct committee
• A (Slack) help desk for technical issues
• A general Slack monitor for questions and engagement
• A Twitter monitor for tweeting about the event
• Session chairs
• Hack Day judges

we also needed the roles below for our online event:

• Zoom participant managers for un/muting, creating/assigning breakout rooms, keeping an eye on hand raising, etc.
• Four sets of Hosts and Co-hosts for the parallel Mini-workshop and demo sessions

What wasn’t initially clear was the effect the government response to COVID-19 had on how staff could participate in the event. Parents now had to factor in homeschooling kids and other caring responsibilities during the day. Other team members had limited or unstable internet, and therefore wouldn’t be able to have prominent Zoom responsibilities. Some members became ill. We therefore ensured that we had two people assigned for each role/duty, as contingency in case someone had to drop out of the event for any reason. This required us to reach out to our network, as there weren’t enough SSI staff members available to cover every role and backup. Luckily, the research software community has no shortage of virtual event experts, so we reached out to our SSI Fellows in the Open Life Science community to help us manage the parallel mini-workshop and demo sessions.

We documented everything to make sure that everyone knew their roles and responsibilities: we created a duties roster, wrote instructions for Zoom Hosts and Co-hosts, created a spreadsheet of the various roles and Zoom room links for the parallel sessions, and provided an index of links to point to all these documents and to help people find anything else related to CW20 that they would need (links to each day’s notes, Zoom rooms, communication channels, session information, etc.).

We organised test calls for CW20 helpers and Zoom Hosts to make sure all the necessary features were enabled for each account (breakout rooms, co-hosts, non-verbal feedback, etc.) and to let the Hosts gain experience using these features. We also scheduled Zoom “Office Hours” for workshop facilitators to join and test the functionality of Zoom in preparation for their sessions and to ask any questions. Finally, we had a Code of Conduct Committee call to walk through the incident response procedure, discuss potential violations that may occur during an online event and any additional risk mitigation we could put in place to prevent them.

CW20 Risk Management

For events such as CW, we typically look at the following risk aspects:

• The main stakeholder and funder (the Software Sustainability Institute)
• Other stakeholders (sponsors, keynote speakers, attendees, session chairs, organising committee members, volunteers/helpers at the event)
• Budget
• Event agenda
• Venue
• Catering
• Communication (with various stakeholders)
• Policies, guidelines and documentation

We admit that a global pandemic was not considered as part of our risk assessment and mitigation plan. We began undertaking our COVID-19 contingency plan at the start of March - before the UK Government had made any official decisions or published any official restrictions to travel and events in response to the virus. We therefore had to plan for scenarios where there were and weren’t official restrictions, which had implications on existing contracts with vendors (venue, catering, accommodation, etc.) and insurance. The three options we considered for CW20 were to carry on with the event in person, move the event fully online, or cancel the event outright. We dismissed the option of a hybrid event - a blend of in-person and online - as we knew we would not have the resources or time to carry this out effectively.

The main stakeholder

The risks that we identified and needed to mitigate relating to the main stakeholder - in this case the Software Sustainability Institute - were:

• Event goals and objectives not aligned with the main stakeholder goals
• Damage to the main stakeholder’s reputation
• Financial losses (which will be discussed below in the Budget section)

In discussion with the main stakeholder, we identified the objectives of CW as the following:

• Topical issues are highlighted, and participants are brought up to speed on them and work towards enablers for them
• Participants come away having met new people, whom they hope to form collaborations with
• It helps improve diversity and representation of voices in this community
• It nurtures SSI Fellows network and other existing communities we interact with
• It’s an enjoyable, welcoming and safe environment
• It raises the profile and reputation of the SSI
• Participants may learn good practice in running similar events
• It improves the interworking of SSI staff at scale

While having an in-person event can strengthen the outcomes of these objectives, they can also all be achieved through a virtual event. Cancelling the event outright would prevent us from achieving many of these objectives.

The following considerations were taken into account regarding the main stakeholder’s reputation:

• If we carried on with the event in person, we risked damaging the Institute’s reputation if participants travelled and became ill (or infected others as a result), or if there was limited attendance and the event did not achieve its objectives.
• If we moved the event fully online, we risked damaging the Institute’s reputation if it was not run well or didn’t meet participants’ expectations.
• If we cancelled the event outright, we risked damaging the Institute’s reputation through our community’s and participants’ disappointment.

We decided to move the event fully online (primarily due to the reasons outlined in the Attendees section below), thus mitigating the damage to the Institute’s reputation through clear and consistent communication with registrants and by managing the expectations of participants.

Other stakeholders

Attendees

In true unconference fashion, the first thing we did was email CW20 registrants a Google Form asking for their input into our contingency plan and to better understand their feelings and preferences about how the event should proceed in light of the rapidly changing COVID-19 situation.

We asked participants what their preferred course of action would be for CW20 in light of Coronavirus (COVID-19): - Option 1: Carry on as planned with the event in person - Option 2: Move event fully online - Option 3: Cancel the event

We received 48 responses, with 37.5% indicating to carry on as planned with the event in person, 47.9% indicating to move event fully online and 14.6% indicating to cancel the event outright.

It was clear that participants preferred not to travel and attend the event in person. If we went ahead with an in-person event, we risked a significant drop-out in attendance, which would be detrimental to an unconference. More importantly - if we went ahead with an in-person event, we risked the health and safety of our participants. This was the primary reason we decided to cancel the in-person event.

Once the decision was made to move the event fully online, we identified the following risks:

• Participants lose travel and accommodation costs
• Not enough participants or many dropping out right before the event
• Too many participants
• Participants complain about a certain aspect of the event (long days, Zoom fatigue, technical issues, general expectations not met)

The strategy we took to mitigate these risks included:

• Provide guidance and documentation to help registrants claim back travel and accommodation costs
• Reduce ticket prices as we would no longer be providing a physical venue and catering
• Close registrations after certain number is reached (~100 in our case) to avoid having more participants than we could manage
• Communicate general guidance and Code of Conduct clearly before and at the event to set the expectations
• Empower participants to come and go from the event as necessary based on their needs

Sponsors

Two questions we considered with regards to CW20 sponsorship were:

• What limits would our sponsors place on event changes before they withdraw or reduce sponsorship?
• Will there be any longer-term implications for getting sponsorship from them in the future?

The risks were that the sponsors would pull out of the event if we made any last minute changes or would not want to partner with the event again in the future. The strategy we took to mitigate these risks was to contact the sponsors immediately after the decision was made to notify them that the event would be moving online and offer to renegotiate any previously agreed packages.

Keynote speakers

The risks identified regarding the keynote speakers were that they might pull out of the event if we made any last-minute changes. The strategy we took to mitigate this risk was to contact the speakers immediately after the decision was made to notify them that the event would be moving online and provide support and opportunities to test their presentation setup before the event.

Risks that we hadn’t considered but that could have been an issue included sickness or being unable to present due to caring or other responsibilities imposed based on lockdown restrictions. We did not have backup keynote speakers in place and would have provided a break during that period or additional networking time in breakout rooms.

Session chairs

The risks identified regarding the session chairs included connection or other technical issues. The strategy we took to mitigate this risk was to have backups for every event role.

Risks that we hadn’t considered but that could have been an issue included sickness or being unable to present due to caring or other responsibilities imposed based on lockdown restrictions. However, our mitigation strategy still applies to these scenarios.

Organising Committee members

The risks identified regarding the Organising Committee members included: - Not kept up to date with the progress of event planning - Not sure how to contribute or how to share information and coordinate between different organising committee teams - Do not know where help is needed and what their duties are at the event - Feel unappreciated or excluded - Some members being unable to help during the event due to insufficient equipment or other implications arising due to lockdown - Members having connection or technical issues preventing them from carrying out their duties

The strategy we took to mitigate these risks included:

• Define different roles and let people choose what they want to do, communicate clearly what is expected from each role
• Have backups for every event role
• Devise a duties roster for the event, where everyone is assigned tasks, and share it ahead of the event
• Thank our Organising Committee members during the event, make sure they are made known to the audience and feel appreciated

Risks that we hadn’t considered but that could have been an issue included sickness or being unable to present due to caring or other responsibilities imposed based on lockdown restrictions. However, our mitigation strategy still applies to these scenarios.

Volunteers/helpers at the event

The risks identified regarding volunteers/helpers at the event included:

• Not having enough volunteers/helpers
• Volunteers/helpers having connection, technical or other issues preventing them from carrying out their duties
• Volunteers/helpers feeling unappreciated or excluded

The strategy we took to mitigate these risks included:

• Reach out to our community and recruit more people with experience organising online events
• Recruit enough volunteers/helpers in order to have backups for every event role
• Communicate expectations clearly and make a duties roster for the event to make sure everyone knows what they are doing
• Make sure they are made known to the audience, thanked during the event and feel appreciated

Budget

As part of the initial discussion with the main stakeholder, we considered the following with regard to our finances:

• What are our committed expenses, lost income (sponsorship, registration), expenses incurred for other things such as refunding Fellows

  • What proportion of people are we supporting to attend?
  • How much will we have to reimburse for travel?

• Are there any additional costs to modify the event?

As part of our contingency assessment, we calculated the financial risk and projected balance for the three scenarios:

1. Carry on in person
   • No change in sponsorship
   • Might lose some registrations
   • No change in what we pay for staff/fellows hotels, although some may now not attend
2. Move CW20 online
   • We would need to seek sponsors’ approval
   • We might lose some registrations
   • We would have to change what we charge from £250 for a full in-person ticket to £50 for an online pass to attend (to go towards invested staff time and prizes)
   • There would be no additional costs as we already had the video conferencing infrastructure in place
   • We could potentially lose a significant amount of money from cancelling the venue, accommodation and reimbursing travel costs that we had agreed
3. Cancel event outright
   • We lose all of the income (registration fees and sponsorship)
   • We pay all of the costs

Event agenda

The risks identified regarding the agenda included:

• Not fulfilling certain plans (such as social activities)
• The days are too long for an online event leading to Zoom fatigue
• Some sessions don’t translate well to an online setting
• Lose nascent quality of unconference event
• Fewer opportunities for networking (for example, no hallway chats)
• Technical issues

The strategy we took to mitigate these risks included:

• Hack the agenda to reduce the length of each day (we removed the social programme to facilitate this and we note that a social element is what participants missed the most)
• Add more breaks into the schedule
• Add an icebreaker session each morning to allow participants to meet new people
• Provide guidance for mini-workshop sessions, although we note to ask for more interaction in the future (and not comprise solely of presentation)
• Reformulate Hack Day and judging process

Venue and catering

Major risks to changing the event at this late stage included the Institute’s reputation with our hosts in Belfast as well as the large financial loss we expected if we cancelled the in-person event (based on the cancellation terms and conditions in our contract).

To mitigate these risks, we arranged a call with the Belfast stakeholders to discuss the situation surrounding COVID-19 and possibly postponing the in-person event to a future date. We were exceedingly grateful that they agreed to postpone to a future date, with no charges for the changes to the room booking dates, audio/visual and catering.

Communication

The risks identified regarding communication included:

• Lack of transparent, efficient and effective communication with different stakeholders; for example, not communicating clearly the changes to the event, expectations, planning decisions, task delegation
• Participants confused about the agenda or not getting information on the right channels and at the appropriate time/frequency
• Too many or not enough pathways to engagement

The strategy we took to mitigate these risks included:

• Update the CW20 website and Eventbrite page as decisions and changes were made (for example, we added a notice about our response to COVID-19, updated the agenda with reformatted programme, made changes to the session descriptions, etc.)
• Consistent emails with registrants communicating any changes made to the event and any updated information regarding refunds, connection details, programme, and general guidance for getting the most out of the virtual event (although we note that we should have provided advice on physical setup, such as having a second monitor for additional screen real estate and a comfortable headset with microphone)
• Documentation for the delivery team (comprised of session chairs, the Organising Committee and volunteers/helpers), such as Zoom instructions for hosts, duties roster, etc. to make sure that everyone was on the same page and knew their responsibilities
• Documented all relevant instructions, links and other information in the participant-facing collaborative notes document to keep everyone synchronised
• Slack workspace for the event that would persist outside of the Zoom meeting for sharing information and participant engagement

Policies, guidelines and documentation

The risks identified regarding policies, guidelines and documentation included:

• Lack of clearly stated policies and guidelines suitable for an online event (e.g. Code of Conduct policy and breach reporting and handling procedures, privacy policy, contribution guide)
• Data privacy issues related to using Zoom and risk of “Zoom-bombing”

The strategy we took to mitigate these risks included:

• Revise Code of Conduct and reporting procedure for online setting and interactions
• Schedule Code of Conduct Committee meeting to make sure everyone understands how to handle reported harassment
• Introduce Code of Conduct Committee at the start of each day
• Ask participants not to publicly share any of the Zoom room links or collaborative documents which contain links to the Zoom rooms and Slack channel

We note that we did not change our privacy policy or apply a license to our collaborative documents; thus to make them available after the event we had to remove personal identifiable information to be compliant with GDPR. In the future, it is worth considering having a privacy policy and license in place to enable easier sharing of these outputs, but this is a discussion for the Steering and Organising Committees and would need to be properly communicated to participants.

CW20 Outputs and Outcomes Plan

Outputs

• Presentations

  • From keynotes (slides, videos)
  • From lightning talks (slides, videos)
  • From mini-workshops and demo sessions (videos)

• Speed blog posts from the discussion sessions

• Collaborative ideas

• Hack Day

  • Information about entries and winners
  • Presentations

• Infrastructure resources

• Blog posts

Outcomes

As part of our post-event feedback survey, we asked participants how confident they were in their understanding of each of the CW20 themes in the context of their research/work, both before and after the event. In general, participants were more confident about their understanding of Open Research, Data Privacy and Software Sustainability after attending CW20.

On average, CW20 improved attendees’ confidence in their understanding of Open Research from 3.6/5 to 4.2/5, a 17% increase.

On average, CW20 improved attendees’ confidence in their understanding of Data Privacy from 3.1/5 to 3.7/5, a 19% increase.

On average, CW20 improved attendees’ confidence in their understanding of Software Sustainability from 4/5 to 4.3/5, an 8% increase.

As CW20 took place online relatively early on in the pandemic, it had an impact on virtual events run by others, and the Institute was often consulted regarding the use of Zoom.

CW20 Closing Down the Event Project Stage

At the end of the event

We made participants aware that we would be gathering feedback, following up with them about speed blogs and their prizes at the end of the event. We thanked the sponsors, volunteers, speakers and workshop facilitators for their contributions towards delivering the event, and we also thanked the participants for engaging under such unprecedented circumstances.

After the event

We emailed participants with a form to collect feedback within a week of the event. We also had a meeting with the delivery team to compile notes on lessons learned from running the event, what worked well and what could be improved in the future.

In the months after the event, we followed up on the speed blogs that were produced as part of the discussion session. We published them along with reflection blog posts, collected and shared the presentation slides, sanitised and shared the event infrastructure resources, processed and published the session recordings, and contributed to various guides for running virtual events. You can see the details of the CW20 outputs in the CW20 Outputs and Outcomes Plan and the timeline of when we made them available in the CW20 Timeline.

When are you done?

All of the outputs that we planned to produce are now available and the event is complete.

Collaborations Workshop 2021 (CW21) - Online

In Practice authors: Rachael Ainsworth (ORCID)

The Collaborations Workshop (CW) is the annual flagship event of the Software Sustainability Institute. It brings together researchers, developers, innovators, managers, funders, publishers, leaders, and educators with a focus on research software to explore and share best practice.

We made the decision from the beginning that the Collaborations Workshop 2021 (CW21) would take place online for two main reasons:

1. It was clear that the global COVID-19 pandemic would be ongoing and there would still be hesitancy from both organisers and participants towards traveling to an in-person event. (It turned out that there were still many restrictions around international travel and in-person events at the time of CW21, so we would have been forced to remain online had we planned an in-person event.)
2. It was an opportunity to improve and build off of what we achieved running CW20 Online.

Here is how we did it. This In Practice write up covers how we organised CW21 Online, following the steps in the Feasibility Stage and the Event Project Stage of the main Event Organisation Guide. Alternatively, if you would like to read about what happened at CW21, that can be found in this CW21 Highlights blog post.

CW21 Feasibility Stage

During the Feasibility Stage , the event idea is being explored more thoroughly. At this stage, various things are needed before a formal sign-off and progression to the Event Project Stage (i.e. before the Institute/main stakeholder agrees to take on the staff effort, financial risk and opportunities afforded by running the event).

The following sections include the information from the Feasibility Stage which is needed for evaluation by stakeholders.

CW21 Goals and Objectives

What we hope to achieve by running the event (goals) and a set of measurable steps towards achieving our goals (objectives).

Themes

Goals

• Increase confidence and understanding of key topics for the research software community that also fit into the wider aims and goals of the Institute.

Objectives

• Participants will discuss topics related to the themes:
  • FAIR Research Software
  • Diversity and Inclusion
  • Software Sustainability
    • Capacity building of skilled researchers
    • Recognition of software contributions
    • The need for software oriented career paths
    • The importance of reproducible research
    • Open development
• FAIR Research Software fits into the Institute’s wider goals and activities around recognition of software as a research output and reproducible research.
• Connect Librarians, Research Data Managers and FAIR Data working groups with the FAIR Research Software community and working groups.
• Participants will learn about the diversity and inclusion research conducted by the Institute and provide feedback.
• Unaddressed discussion topics related to the themes will be brought into the SSI Fellow’s Community Calls.

Social

Goals

• Improve social aspects of a virtual unconference.
• Increase opportunities for participants to network, meet new people and start new collaborations.
• Increase opportunities for community members who are unable to attend to engage with the discussions.

Objectives

• Call for participant-led social programme.
• Dedicated networking session.
• Participants to have met at least 7 new people.
• Participants intending at least 1 new collaboration on average.
• At least 25% who have not attended CW before and are not Fellows or in the Fellowship application process.

Diversity

Goals

• Improve accessibility from CW20.
• Increase diversity and representation from CW20 (race, gender, career stage, role, location, etc.).
• Increase opportunities for community members who are unable to attend to engage with the discussions.

Objectives

• Include closed captioning / transcription services.
• Provide advice to participants on how to get the most out of the virtual unconference (setup, equipment, expectations).
• Include Diversity and Inclusion as one of the themes for CW21.
• Ensure at least 50% of the keynote speakers are women.
• Increase BAME representation above CW20.
• Measure and compare demographics of CW21 with CW20.
• Develop social media strategy to promote discussion topics which arise during the unconference to include and engage community members who are unable to attend (e.g. due to location or other commitments).

Sponsorship

Goals

• Maintain relevance of sponsors to themes and audience.

Objectives

• Reformulate sponsorship packages to account for online format.
• Target new sponsors compared to previous years (esp. Silver and Gold).
• Invite sponsors from previous years to engage.

Technical

Goals

• Improve the documentation and infrastructure for running a virtual unconference.
• Facilitate a smoother running of the Hack Day compared to CW20.
• Increase reusability and openness of unconference outputs.
• Increase participant interaction in mini-workshops and demo sessions.

Objectives

• Document process and lessons learned as we go along.
• Successful use of uCONFLY infrastructure.
• Restructure Hack Day
• Allow free form team formation in the Hack Day more akin to a face to face gathering.
• Design adjustable schedule depending on number of participants/teams.
• Offer facility for teams to break into sub-groups to work on specific tasks.
• Write clear guidance for what information teams should include in their project demos to make judging more efficient.
• Use platforms appropriate for the sessions (investigate use of platforms for online mingling).
• Participants take skills, techniques, tools and resources back to their communities.

CW21 Audience

The Collaborations Workshop series brings together researchers, developers, innovators, managers, funders, publishers, leaders and educators to explore best practices and the future of research software. As such, our main target audience includes changemakers, advocates and ambassadors of all career stages and identities for each of these stakeholder groups who will continue and build off of the conversation and collaborations beyond the workshop. Our target audience comprises of people who can dedicate time to a three-day event as this will allow us to meet our CW21 Goals and Objectives. By being multi-disciplinary, we can engage a wide range of people who can bring knowledge and skills back to their domains.

The geographic location of the CW target audience is the UK/Europe. As CW21 will be online, the target audience’s geographic location will not be limited except by time zone - participants in Europe, Africa and eastern timezones of North and South America are more likely to participate in a 10:00-16:00 BST event. This will give us the opportunity to invite changemakers who would not normally be able to travel to the UK, and make the event more accessible. We also aim to engage people in other timezones in the discussions through social media and sharing of outputs.

Below we discuss specific audiences, communities and job roles to target related to the themes of CW21. We will encourage connections within these communities to reach out to their communities so that the recommendation/promotion comes from a trusted member and is more likely to be actioned.

FAIR Research Software

• Funders
• Policy makers
• Publishers
• Librarians
• Research Data Managers
• FAIR projects (not necessarily software-focused)
• Open Science/Research advocates

Example communities

• NL eScience Centre
• UoM eScience Lab
• RDA FAIR4RS Working Group
• FAIRsFAIR
• FAIRsharing
• FAIRplus

Diversity and Inclusion

• Community Managers
• Recruiters
• Policy makers
• Funders
• Grassroots groups

Example communities

• The Carpentries
• CSCCE
• UKRI
• Wellcome
• WHPC

Software Sustainability

• Developers
• Researchers
• Publishers
• Funders

Example communities

• RSE (UK, US, NL, DE, etc…)
• eLife
• JOSS

CW21 Date

The date needs to fit into the calendar of other events that are likely to affect the CW21 Audience. Things to consider here may include being outside of University term time and preferably not clashing with school holidays for those with caring responsibilities. Bank holidays and religious holidays should also inform the decision.

The best dates for CW21 were Tuesday, 30 March to Thursday, 1 April 2021. For publicity and planning purposes, we noted that this was the first week of British Summer Time (BST).

CW21 Venue

Due to the ongoing COVID-19 pandemic, it was decided that CW21 would take place virtually. CW21 followed on from CW20 which also took place virtually. This largely informed our CW21 Goals and Objectives and CW21 Audience decisions.

We compiled and discussed various platform options, but Zoom was still the preferred choice, especially in light of recent updates (e.g. participants being able to choose their own breakout rooms) and that we already had professional subscriptions.

Using additional platforms for networking and the social programme was explored during the CW21 Event Project Stage.

CW21 Outputs and Outcomes

Outputs are the things produced during or for the workshop, be it physical or virtual, and outcomes are the changes made possible due to running the event. These should be linked to the event goals. Some of these will be the same as the objectives of the workshop. They should include the type of resources that will be available after the event. That is, whether a report will be written, videos of the sessions or any other resources will be made available.

Outputs

• Recordings of the sessions shared via the SSI Youtube channel
• Closed captioning, transcriptions
• Blog posts from the discussion session
• All slides and other outputs shared via the conference portal (e.g. Figshare)
• A highlights blog post on the SSI website
• A reflections blog post on the SSI website on how we ran it
• Collaborative notes documents and other unconference infrastructure resources shared
• Feedback into Event Organisation Guide (e.g. as a case study)
• Any sponsor outputs (e.g. Tweets, videos, etc.)

Outcomes

• Facilitation of new collaborations between participants (e.g. increased participation in FAIR 4RS working group)
• Outputs are more widely shared, increasing impact of discussions and visibility of CW/SSI
• Increased accessibility and inclusion
• Community members who are unable to attend the event are still able to engage with the discussion
• Impact on virtual events run by others through influencing and informing how they run their virtual events and reuse of infrastructure resources
• Ease of running a virtual CW again in the future

CW21 Event Project Stage

The idea was accepted and the constraints and practicalities have been signed off. Now we are in the Event Project Stage - it’s time to organise and deliver the event itself.

The Event Lead needs to think about infrastructure, setting a timeline, committees and roles needed in the run-up to the event, the details of the agenda, publicity, how to go about attracting sponsorship, event roles on the day and even how to close down the event.

The following sections go into these elements of CW21 in more detail.

CW21 Infrastructure

As our primary CW21 Goals and Objectives were to build off of CW20 and focus on improving aspects around socialising, networking, accessibility and inclusion, we kept much of the CW20 Infrastructure. However, we also ended up using many additional platforms to achieve our goals.

Platforms

We used the following platforms to help us organise and run the event.

Event management

• GitHub: We used a private GitHub repository to project manage the event, using Issues and Project boards to keep track of what needed to be done and our progress.
• SSI website: The CW21 event page was hosted on the Institute website.
• Mailchimp: We used Mailchimp to set up a registration of interest form to notify our audience of CW21 updates and when event registration opened.
• Eventbrite: We created a CW21 Eventbrite page to manage registrations, fee processing and send email notifications to registrants.
• Bit.ly: We generate a lot of links during a CW and we used Bit.ly to manage them, make them more user-friendly and monitor engagement/clicks.
• Google Forms: We conducted the calls for sponsorship, financial assistance, lightning talks, mini-workshop and demo sessions, and participant-led social programme activities using Google Forms.
• Figshare: Through our partnership with Figshare, we were once again able to manage conference outputs such as lightning talk slides through a Figshare conference portal.
  • Lightning talk presenters, mini-workshop facilitators, and participants who wanted to generate DOIs for their CW21 outputs were able to submit directly to the portal using this form.
• uCONFLY: We used the SSI’s unconference resource management system uCONFLY to manage collaborative Google documents for the interactive sessions.
  • Following our procedure from CW20, we used uCONFLY purely as an admin tool by creating the templates for the Discussion Session, Collaborative Ideas session, Mini-workshop and demo sessions, and Hack Day ideas, and generated all of the needed instances ahead of time ourselves.
• Google Sheets: We collated the uCONFLY generated documents via Google Sheets (see this example spreadsheet) which we linked to in each day’s collaborative note-taking document/agenda (see section on Collaborative Documenting below for details).

Video conferencing

• Zoom: We used Zoom as our primary video conferencing and management platform, as we had the most experience with it and it had the best performance, scalability and functionality out of the many platforms we tested.
  • We attended many of the CSCCE Tools Trials which tested and compared features from a variety of virtual conferencing platforms, including for QuiqoChat, Etherpad with video and Wonder, and found that Zoom still best met our needs.
  • The ability to create and manage breakout rooms is essential for our interactive sessions such as the Discussion session, Collaborative Ideas session and Hack Day.
  • The added functionality in Zoom allowing participants to self-select their own breakout rooms was a game changer (we did not have this for CW20, which made breakout room assignments tedious).
  • We had six professional/education Zoom Host accounts in total to accommodate parallel sessions: one for the main/plenary room (where the Discussion session, Collaborative Ideas session and Hack Day were managed in breakout rooms), plus five for the mini-workshop sessions which took place in parallel outside of the plenary room.
  • We also attended a Zoom safety drill hosted by the CSCCE to test out the various security features available on Zoom, and to role play what it would be like to have to report and evict a bad actor and then deal with the fallout.
• YouTube: One of our goals was to increase opportunities for community members who were unable to attend the event to engage with the discussions, and one way we facilitated this was by live streaming the keynote presentations and panel via the SSI YouTube channel.
  • YouTube was also used to share the session recordings after the event.
• Restream: We needed to stream from Zoom to both YouTube and our live transcription service Otter.ai (see Accessibility section below) simultaneously, which required the use of a third party streaming application (as Zoom could only stream to one service at a time).
  • We followed this guide on Captioned video calls AND YouTube streaming by Open Life Science to get the three platforms to work together (and practiced the procedure many times ahead of the event).
• RemotelyGreen: RemotelyGreen was used for a speed networking social hour as part of our social programme.
• GatherTown: GatherTown was used for a quiz as part of our social programme.
• Mozilla Hubs: Mozilla Hubs was used for a virtual meetup as part of our social programme.

Accessibility

• Otter.ai: We used Otter.ai for live transcription and captioning during the event, as well as to generate subtitles for the session recordings in order to improve accessibility.
  • This required Otter.ai Business subscriptions for each Zoom Host account used in running the event.
  • Some feedback we received during the event, was that participants appreciated the live transcription because it meant that they were able to focus on the presentations without having to take their own notes.

Chat system

• Slack: We used Slack as our primary chat platform, as it persists outside the meeting room which is helpful for maintaining access to resources shared, referencing discussions and facilitating community engagement.
  • We created practical channels for event organisation and communication, such as a #help-desk channel for participants to direct their technical issues to and #introductions for participants to introduce themselves.
  • Private backchannels were created for the Code of Conduct Committee, Zoom hosts, SSI staff, and Hack Day judges.
  • Participants were able to create their own channels for their own use such as for their Discussion session groups and Hack Day teams.
  • Because of their success during CW20, we created more informal channels such as #pets-at-cw21, #kids-at-cw21 and #outside-my-window-cw21 for sharing photos of pets, children and working from home environments during lockdown.
• Sli.do: We obtained a professional subscription of Sli.do to offer another pathway for audience Q&A and engagement, which added an interactive element for participants and also allowed them to ask questions anonymously (and moderated by the organisers).

Collaborative documenting

• Google Docs: We used Google Docs as our primary platform for collaborative note-taking and keeping everyone synchronised.
  • We created a main document for each day which contained that day’s agenda (which linked to other sections of the doc with information and space for note-taking and questions for each session), connection details and other important links, participation guidelines, guidance and instructions, roll call, space for feedback, and a disclaimer in the header asking participants not to share any links publicly during the event (see the collaborative document for CW21 Day 1 here).
  • We added a license (CC BY 4.0) to all of the Google Docs generated for CW21 so that participants would know how they could be used going forward.
  • Although we chose Google Docs because it has a low barrier to accessibility (no need to log in or register and you do not need to know specific syntax or markdown language), participants using screen readers or voice control had difficulty using it (we empowered them to use any platform that worked best for their needs, and HackMD seemed to work better for these cases).

Resources

We created the following infrastructure resources to help us manage the event.

Organiser-facing

• A spreadsheet for planning the event and keeping track of registrations and budget.
• An index of links to documentation and event resources for organisers to easily navigate on the day.
• A duties roster with assignments, lists and descriptions of the roles and responsibilities.
• A spreadsheet for the mini-workshop and demo sessions assignments with links to the Zoom rooms, notes documents, and live transcriptions for the parallel sessions.
• Zoom instructions for the hosts and co-hosts of the parallel sessions.
• A checklist for the event chair of what tasks need to be done before and at the start of the event (such as a reminder to send the connection details, assign co-hosts, record the event and take a group photo).
• A checklist for the event chair of how to set up the live stream during the event (a complicated procedure involving steps in Zoom, YouTube, Otter, and Restream).
• A speaker guide to help the keynote speakers with their recordings.
• A spreadsheet to facilitate Hack Day judging and scoring.
• A spreadsheet for keeping track of prizes and winners.
• Instructions for processing the session recordings, generating subtitles and publishing the videos on YouTube.
• A spreadsheet to facilitate and track processing the session recordings (for example, with information on the start and end times for splitting recordings, locations of cover and subtitle files, session information, and YouTube metadata).

Participant-facing

• Collaborative notes documents for each day of the event (with agenda) to guide participants through the event, provide another pathway to engagement and compile questions, notes and outputs from the event:
  • CW21 Day 1 Notes
  • CW21 Day 2 Notes
  • CW21 Hack Day Notes
• Collaborative notes documents for the icebreaker sessions with prompts and space to share responses.
• A template document for the Discussion group session and 26 instances generated in uCONFLY.
• A template document for the Collaborative Ideas session and 26 instances generated in uCONFLY.
• A template document for the Mini-workshop and demo sessions and 10 instances generated in uCONFLY.
• A template document for the Hack Day pitches and 26 instances generated in uCONFLY.
• A spreadsheet to collate the Discussion group session topics and documents, and facilitate group sign-up.
• A spreadsheet to collate the Collaborative Ideas session documents and facilitate voting.
• A spreadsheet to collate the Hack Day pitch documents.
• A spreadsheet to facilitate Hack Day teams registration and sign-up.
• A feedback form.

CW21 Timeline

Event dates: Tuesday, 30 March to Thursday, 1 April 2021

Feasibility stage

Month -8: August 2020

• Draft CW21 Goals and Objectives
• Draft CW21 Venue
• Draft CW21 Date
• Draft CW21 Audience
• Draft CW21 Budget (this has not been made available as it contains information provided in confidence)
• Draft CW21 Outputs and Outcomes

Month -6: October 2020

• Gateway Review with Main Stakeholder to see if the event can go to the Event Project Stage (Approved)

Event Project Stage

Month -7: September 2020

Committees

• Steering
  • Reach out to possible members and let them know about commitments and role
  • Create Doodle Poll for 1st meeting
  • Schedule 1st meeting
• Organising
  • Reach out to possible members and let them know about commitments and role

Publicity

• Draft publicity plan
  • Who announcements are made to
  • When they are made

Month -6: October 2020

Website

• Create event landing page
• Create a registration of interest form
• Draft Sponsorship packages

Committees

• Steering
  • Draft 1st meeting notes and agenda
  • Email 1st meeting notes and reminder to members
  • 1st meeting
  • Invite additional members to committee as needed
  • Create Doodle Poll for 2nd meeting
  • Schedule 2nd meeting
  • Email update of any major progress

Registration

• Design information gathering
• Pre-test registration questionnaire
• Update Code of Conduct
• Draft Intellectual Property Policy
• Update Privacy Policy if needed
• Combine Code of Conduct, Privacy Policy, Intellectual Policy and Social Media Policy into a set of Participation Guidelines
• Send updated policies to Main Stakeholder for comment and approval

Publicity

• Add ‘register your interest’ to publicity list

Agenda

• Draft agenda
• Keynotes
  • Compile shortlist of keynote speakers
  • Email possible keynote speakers: FAIR Research Software

Month -5: November 2020

Committees

• Steering
  • Draft 2nd meeting notes and agenda
  • Email 2nd meeting notes and reminder to members
  • 2nd meeting
  • Create Doodle Poll for 3rd meeting
  • Schedule 3rd meeting
  • Email update of any major progress
  • Provide free registration

Registration

• Open registration
• Email SSI Staff and Fellows instructions for registration

Publicity

• Announce when registration is open
• Publish news items highlighting keynote speaker: FAIR Research Software

Agenda

• Call for mini-workshop and demo sessions
• Call for participant-led social programme activities
• Email possible keynote speakers: Diversity and Inclusion
• Email possible panellists: Diversity and Inclusion

Month -4: December 2020

Agenda

• Diversity and Inclusion panellists established

Committees

• Steering
  • Draft 3rd meeting notes and agenda
  • Email 3rd meeting notes and reminder to members
  • 3rd meeting
  • Email update of any major progress

Publicity

• Publish news item announcing financial assistance
• Publish news item on call for mini-workshop and social activity proposals

Registration

• Open call for financial assistance scholarships

Month -3: January 2021

Agenda

• Diversity and Inclusion Keynote speaker established
• Deadline for mini-workshop and demo session proposals
• Deadline for participant-led social programme activities
• Email speakers and panellists to arrange meetings to discuss details of presentations
• Meeting with Diversity and Inclusion Panel Chair
• Meeting with Diversity and Inclusion panellists to finalise format and content

Committees

• Steering
  • Create Doodle Poll for 4th meeting
  • Schedule 4th meeting
  • Email update of any major progress

Publicity

• Submission deadline reminder news item

Registration

• Start weekly update to planner document from registrations
• Review financial assistance applications submitted in December

Sponsorship

• Work through Sponsorship spreadsheet and invite organisations to partner

Month -2: February 2021

Agenda

• Call for lightning talks
• Review mini-workshop and demo session proposals
• Review participant-led social programme activity proposals

Committees

• Steering
  • Draft 4th meeting notes and agenda
  • Email 4th meeting notes and reminder to members
  • 4th meeting
  • Create Doodle Poll for 5th meeting
  • Schedule 5th meeting
  • Email update of any major progress

Publicity

• Lightning talk registration news item
• Diversity and Inclusion Keynote speaker announcement news item
• Diversity and Inclusion Panel announcement news item
• Mini-workshops announcement news item

Website

• Update Agenda
• Create Discussion Groups page
• Create Collaborative Ideas page
• Create Lightning Talks page
• Create Hack Day page
• Create Speakers page
• Create Mini-workshops/demos page

Month -1: March 2021

Agenda

• Compile discussion group topics
• Confirm lightning talk schedule
• Confirm networking and social sessions
• Full agenda now available

Committees

• Steering
  • Draft 5th meeting notes and agenda
  • Email 5th meeting notes and reminder to members
  • 5th meeting
• Code of Conduct
  • Confirm committee members
• Organising
  • Duties roster and signup

Communication

• Contact mini-workshop facilitators with guidelines
• Contact lightning talk speakers with guidelines for submission
• Email registrants 1 week in advance of any actions
  • Connection details
  • Discussion topics
  • Any social programme sign-ups necessary
  • Space for lightning talks
• Email registrants 1 day in advance of any actions

Event run up actions

• Obtain keynote talk recordings, process and transcribe
• Plan breakout groups distribution
• Ongoing: update the CW planner spreadsheet
• Assigning activities and roles to SSI Staff in advance
• Test infrastructure (transcription, live streaming, Zoom settings)
• Construct feedback form draft (pre-test/review if possible)

Hack Day

• Reach out to possible Hack Day judges
• Order prizes

Infrastructure

• Procure transcription / captioning licenses
• Procure any additional platforms as needed (Sli.do Professional, Restream Professional)
• Set up Figshare conference portal
• Create collaborative notes documents for:
  • Day 1
  • Day 2
  • Hack Day
• Setup uCONFLY instance for management of discussion groups, collab ideas and hack day ideas
  • Discussion session templated docs
  • Collaborative Ideas session templated docs
  • Mini-workshop and demo session templated docs
  • Hack Day Ideas templated docs
• Spreadsheets for managing
  • Discussion session
  • Collaborative Ideas session
  • Mini-workshops and demos
  • Hack Day

Resources

• Create Chair slide deck

Publicity

• Blog post/news item announcing Platinum sponsor
• News item promoting platform sponsor

Website

• Create Accessibility page
• Create Social Programme page
• Update Lightning Talks page

Event Running Stage

Month 0

• Run event as planned
• Have a Hack Day judges meeting
  • (Re)visit judging criteria

Post event Stage

Month 1: April 2021

Agenda

• Updated the agenda with links to resources

Feedback

• Emailed participants with a feedback form within 2 weeks after the event
• Sent a reminder email a few weeks later
• Held a feedback prize draw to thank them for taking the time

Resources

• Promoted the use of the Figshare conference portal to share CW21 outputs
• Chased speakers for any un-captured resources
• Collected Zoom recordings from the Hosts/Co-Hosts of each session

Month 2: May 2021

Publicity

• Published a session facilitator’s experience of their CW21 mini-workshop
• Published a blog post with highlights from the event

Resources

• Contacted speed-blog authors and connected them with the Communications Team for publication to the SSI blog
  • 3/16 discussion groups’ speed blogs published
• Prepared videos for editing by
  • Organising files
  • Updating processing instructions for Events Team
  • 23/43 videos processed

Month 3: June 2021

Publicity

• Published another session facilitator’s experience of their CW21 mini-workshop

Resources

• 43/43 videos processed
• 6/16 discussion groups’ speed blogs published
• Began working on the EOG In Practice: CW21

Website

• Updated language of event website to show that it was in the past
• Create Collaborative Ideas groups page
• Create Hack Day groups page

Month 4: July 2021

Publicity

• Announced availability of session recordings

Resources

• 8/16 discussion groups’ speed blogs published

Month 5: August 2021

Resources

• 11/16 discussion groups’ speed blogs published
• First draft of the EOG In Practice: CW21 submitted for review

Month 8: November 2021

Resources

• First draft of the EOG In Practice: CW21 reviewed and submitted to Communications Team for final edits and promotion

CW21 Budget

As we were planning for the event to be virtual from the beginning, we were able to budget for our main expenses out of our expected income from registrations and sponsors. We kept the registration fee at £50 (or £25 for access to a single day) to ensure we could cover the costs of prizes (which are a CW tradition) and live transcription and captioning which would be new for our event. We estimated that there would be around 100 registrations, and that we would have two or three Silver sponsors. As we began to receive more sponsorship, we were able to make decisions on additional costs such as procuring additional platforms to enhance our event as well as offering more financial assistance to participants.

Any profit that is made from CW allows the SSI to fund special collaborators and sponsor other events and activities.

Existing infrastructure

• Six Zoom professional or education accounts

Income

• Registrations

• Sponsorship

  • 3 Silver Sponsors
  • 1 Platinum Sponsor
  • 1 Platform Sponsor (RemotelyGreen)

Expenses

• Eventbrite fee
• Otter.ai business accounts (2 annual education accounts and 4 one-month business accounts)
• Sli.do professional account
• GatherTown premium plan for quiz as part of the social programme
• Restream professional account to stream from Zoom to both YouTube and Otter.ai
• Prizes for First, Second and Third place Collaborative Ideas groups
• Prizes for First, Second and Third place Hack Day teams
• Prizes for Feedback raffle
• Speaker fees and gifts
• Registrations for invited speakers, Steering Committee members, and other key invitees
• Financial assistance for participants

CW21 Committees

Steering Committee

The CW21 Steering Committee comprised of representatives from the Institute, the workshop sponsors, experts in virtual event organisation, and experts in FAIR Research Software, Diversity and Inclusion, and Software Sustainability (the workshop themes). We held monthly meetings where members helped to guide the agenda and direction of the workshop (recommending speakers and session types), provided advice for running the event (recommending platforms and how to improve accessibility) and contributed to many of the workshop sessions.

Organising Committee

The CW21 Organising Committee comprised of representatives from the Institute. Members had more regular communication, with a focus on logistics and identifying and being responsible for roles/duties needed to run the event. Organising Committee members worked to

• Create and test the infrastructure for running the virtual event
• Help with the event publicity
• Help run the event on the day

Code of Conduct Committee

The CW21 Code of Conduct (CoC) Committee comprised of three members who were Institute staff and two external. There was a mix of gender, race and career stage within the Committee to provide a range of options for participants to choose who they felt most comfortable reporting to. Two members of the Committee had participated in Code of Conduct Incident Response training.

Before the event, we sent the following to CoC Committee members to read through and raise any issues or questions:

• The Code of Conduct
• The Attendee Procedure for Reporting Harassment
• The Staff Procedure For Handling Reported Harassment

All of these materials were made available within the CW21 Participation Guidelines which participants had to have read and agreed to abide by at registration.

During the event, we had a private Slack channel in the CW21 Slack workspace for CoC Committee members in case any incidents arose. No misconduct was reported from CW21.

CW21 Agenda

As we were able to organise CW21 as an online event from the beginning (versus having to transition from an in-person event to an online event within three weeks like we had to do for CW20), we were able to be very intentional about the agenda and programme to achieve our CW21 Goals and Objectives.

One of our primary goals was to increase opportunities for socialising and networking which was sorely missed at CW20. We also wanted to make sure that we minimised the length of the day and maximised breaks to combat Zoom fatigue. In general, we had many session types with shorter lengths in order to keep participants engaged. Breaks were again strategically scheduled to allow time for the creation of breakout rooms for the next session.

Informing

Keynote Talks

For CW21, we had two keynote presentations - one for each new theme (FAIR Research Software and Diversity & Inclusion). The keynote presentations were pre-recorded by the speakers. This was done so that the talks could be transcribed and captioned ahead of time, to mitigate risk of connection failures on the day, and also because both keynote speakers happened to be located in time zones incompatible with the event. The length of each keynote session was 30 minutes, with the suggested format of up to 20 minutes for the presentation and the remainder for introductions and questions.

The Chair of the session introduced the speaker live and reminded the participants that they could take notes and ask questions in the collaborative notes or via the Sli.do website/app, and that this session would be live streamed via YouTube. The Event Lead choreographed sharing the pre-recorded presentation via Screen Share in Zoom and live streaming the session on YouTube (following this guide on Captioned video calls AND YouTube streaming by Open Life Science to get Zoom, YouTube and Otter.ai to work together via Restream - see CW21 Infrastructure for more information).

One keynote speaker was able to attend the event and answer questions live, which was facilitated by the session Chair who relayed questions submitted via Sli.do. For the case where the second keynote speaker was unable to be present, we compiled the questions from Sli.do and emailed these to the speaker to be answered in their own time. The answers were then shared with participants via the collaborative notes document.

You can view the live streamed keynote presentations here.

Lightning Talks

Lightning talks provide the perfect opportunity for participants and sponsors to introduce themselves at the workshop. CW lightning talks are two minutes for one slide, and can be used to showcase a project, introduce a problem, or promote a mini-workshop or demo session later in the programme.

This year, we conducted a call for lightning talks with a short Google Form to help with planning and prioritising new voices (in previous editions of CW, it was first come, first served for lightning talks). Confirmed lightning talk presenters were then asked to submit their slides to the Figshare conference portal ahead of the event following these instructions.

We then compiled all the slides into a single slide show to maximise efficiency during the session, and integrated a two-minute timer within each slide to help keep speakers to time and minimise having to cut speakers off. The session Chair shared the slides via Screen Share in Zoom, and invited each speaker to unmute when it was their turn and deliver their lightning talk. The Chair also enabled Spotlight Video for the speaker so that they were visible to all participants and in the recording.

Exploring

Panel

Because the theme of Diversity and Inclusion is so expansive, one of our Steering Committee members suggested that we also have a panel on this theme to include a variety of perspectives relating to disability and accessibility in research software. The panel session was 45 minutes in length, with three minutes for the Chair to introduce themself and set the scene for the discussion, three minutes for each panellist to introduce themselves, and 30 minutes for questions and answers. The panel met ahead of the event to discuss what each panellist wanted to highlight and bring to the discussion, as well as to come up with a set of pre-prepared questions.

The panel session took place live during the event (not pre-recorded), and was also live streamed on YouTube. The Chair facilitated the discussion using a mix of the pre-prepared questions and questions submitted by participants via Sli.do. If questions were not able to be answered during the session, the panellists answered them in writing after the session.

You can view the live streamed panel discussion here.

Discussion Groups

The Discussion session allows groups of people to discuss a topic that interests them in a way that furthers our knowledge of that topic. This session is a fundamental part of the Collaborations Workshop: it helps people learn about new ideas and work together on solving shared problems. The output of the discussion session is a speed blog post from each group to help disseminate the insights to the wider community.

The session was conducted the same way as during CW20 except that we were able to take advantage of the new Zoom feature that allowed participants to self-select their own breakout rooms.

Participants suggested and signed up to the topics that they wanted to discuss in a spreadsheet (example spreadsheet). This year, in addition to assigning each discussion group a Group ID, they were also assigned a more memorable Group Name. The Group IDs were once again the letters of the alphabet, and the associated Group Names were fruit names that began with that letter. For example, Group ID A had Group Name Apple, Group ID B had Group Name Banana, and so on. Breakout rooms were then created (with the option selected to allow participants to choose their rooms) and renamed to “<Group ID> - <Group Name>”, for example “A - Apple”. This not only made it easier for individuals to remember and find their group’s breakout room, but it was also more fun! The groups were directed to their discussion group reporting document via the “Groups” tab in the spreadsheet, in which they worked together on their speed blog posts.

Collaborative Ideas

The Collaborative Ideas session is used to get people talking about their work. They identify problems within research software and come up with a solution, usually a project that can be taken forward to the CW Hack Day.

The session was facilitated in a similar way to the Discussion session except that the group assignments for the breakout rooms were randomly generated. This time the Group Names were based on famous cats (Asparagus, Bagpuss, and Felix to name a few), so the group that was assigned to breakout room “Asparagus” retrieved their Collaborative Ideas document for Group Name Asparagus in the associated spreadsheet (example spreadsheet), and so on.

Voting was again facilitated in a “Voting” tab of the spreadsheet, where participants where asked to simply add their names below the ideas that they wanted to vote for. The votes were tallied at the end of Day 2, and the teams with the top three ideas were awarded prizes. This year, the prizes were Redbubble digital gift cards, which give the recipients the choice of millions of designs by independent artists printed on a range of products. Redbubble are committed to social responsibility and sustainability, and digital gift cards provide a more international- and pandemic-friendly prize for our participants.

You can read more details about how we run the Collaborative Ideas session in the Collaborations Workshop section of the CSCCE’s guide to Using virtual events to facilitate community building: event formats.

Creating

Hack Day

CW21 concluded with the Hack Day, where teams formed to work on projects generated during the Collaborative Ideas session and other ideas pitched during the course of the event.

Hack Day project pitches and team formation took place at the end of Day 2. Participants took turns promoting a project to attract potential team members, and then breakout rooms were opened for teams to form around the proposed projects. Many of the Hack Day projects came from the Collaborative Ideas session. A spreadsheet was used for teams to register their projects and members, assign Zoom breakout rooms and Slack channels, as well as collect project repositories.

One Zoom room was used to facilitate the Hack Day, with three breakout rooms created for each team, one for the judges, and a few extra side rooms for good measure (which was a lesson learned from CW20 - to give more spaces for meetings between team members and side meetings between people in general). The new Zoom feature that allowed participants to choose and move around the breakout rooms without having to assign them as co-hosts made the Hack Day significantly more smooth than at CW20. A schedule was scaffolded so that participants could work around suggested break times and judges’ visits, and be ready in time to demonstrate their projects.

Each Hack Day team had five minutes to demonstrate their projects and how they met the judging criteria:

1. Novelty, creativity, ‘coolness’ and/or usefulness
2. Implementation and infrastructure
3. Demo and presentation
4. Project transparency
5. Future potential
6. Team work

The judges, comprised of Institute Staff, Fellows, collaborators and CW21 Sponsors, then went into a breakout room to deliberate. The teams that came in the top three places were also awarded Redbubble digital gift cards.

Teaching

Mini-workshops and Demos

Mini-workshops and demo sessions give an in-depth look at a particular tool or approach and a chance to query developers and experts about how this might apply to participants’ areas of work.

For CW21, we set a limit of 10 mini-workshops across two 30-minute sessions. 30 minutes does not feel like a long time for a workshop or demonstration, so we highlighted to facilitators that it was important to plan how they will use the time. We suggested that incorporating a combination of presentation and collaborative group activities, discussions, audience Q&A or feedback, are great ways to engage a virtual audience, and to provide information about how participants can become involved in follow-up activities.

We used a separate Zoom account for each of the five parallel workshops in each session, with an assigned Host (and Co-host for backup) for handling the logistics of the mini-workshop (managing Zoom, recording the session, pointing to the relevant notes document) and helping the workshop facilitator with any technical issues. We generated templated Google Docs for each session that the facilitators were empowered to adapt for their use.

We directed participants to the different Zoom room links in the day’s collaborative notes document and in the Slack channel, and someone stayed in the main room to help people get to where they wanted to go.

Networking

Pre-event Social Hour

A speed networking social hour was sponsored and hosted by RemotelyGreen the evening before the official start of the event. It was designed to facilitate connections between CW21 participants with similar interests and goals. Participants were matched in groups based on selected topics of interest, and provided an icebreaker prompt to help guide the interaction.

The connection details were emailed to participants, who needed to set up an account to join the event (this was a quick process, either using an existing LinkedIn account or an email address). The platform then took care of moving participants through speed networking encounters, allowing them to choose which topics they would like to talk about.

Some of the feedback received included, “[I] talked to lots of new people tonight, probably more than if the workshop was in person.”

We also made a Zoom room available in parallel at the same time as an alternative option.

Morning Coffee Chats

Multiple breakout rooms were made available within the main Zoom room 30 minutes before the official start of days one and two for participants to mingle, connect and chat over their morning brew. Some breakout rooms were used, but we found that most participants who joined stayed together in the main room, where we did a round robin for people to introduce themselves and respond to a prompt.

Icebreakers

We started each day with an icebreaker to get participants used to the infrastructure and to have a chance to chat in small groups. We asked a couple of yes or no questions for participants to respond to using Zoom’s non-verbal feedback feature, and then randomly assigned them into breakout rooms of 3-4 people to discuss the following prompts which we thought would be both fun and useful:

• What are you reading, watching and/or listening to at the moment (and why)?
• What advice/resources/tools/guides do you find useful for working remotely (and why)?

We then asked participants to write down their answers in the note-taking document during a few minutes of “silent documenting”, which had the added benefit of empowering people to directly contribute to the shared notes (example icebreaker document).

Lunch and Coffee Breaks

We encouraged participants to use the breaks to take time away from their screens, but kept the main Zoom room open in case they wanted to talk with other people. We opened a number of breakout rooms as side rooms for participants to have conversations away from the main room and as sponsor booths for participants to chat with and learn more about the CW21 sponsors. We also had an interview room where our Communications Officer interviewed participants about their experiences to use as promotion for future events.

Dedicated Networking Session

A dedicated 30-minute networking session was built into the main programme to encourage participants to network with each other, as we expected a large drop off in attendees to the social programme activities.

We adapted this Recipe for rOpenSci’s Unconf Ice Breaker for our community. The objective of this icebreaker was to facilitate connections between participants based on commonalities. The session Chair stated a prompt, and then asked participants to move into the breakout room that resonated with their response to that prompt. Participants had 10 minutes to talk with the people in their breakout room to introduce themselves, discuss, and share opinions around the prompt. When participants returned to the main room, the Chair asked for volunteers to introduce themselves to the entire group and share their opinions or response to the prompt.

The first prompt was, “My favourite programming language is…” and we created breakout rooms named “C/C++”, “Python”, “R”, “Other”, and “I don’t code”. This was intended to be a lighthearted and simple prompt to get everyone warmed up during the exercise.

The second prompt was more serious: “I know where I fit as a member of the research software community. For example, I have a clearly defined role and career path, and am confident about where I contribute within the research software ecosystem. (Agree vs disagree)” with breakout rooms named “Strongly agree”, “Somewhat agree”, “Neither agree nor disagree”, “Somewhat disagree”, and “Strongly disagree”. This prompt was intended to highlight that there are many pathways and emerging roles within research software, and not everyone knows where or how they fit.

Evening Quiz

At the end of Day One, a participant facilitated a quiz in GatherTown where other participants tested their general knowledge in various categories, including software and academia-related rounds. We chose a setup so that the facilitator was spotlighted on a platform so that all participants could hear and see them as they read out the questions. We also had private spaces as tables for participants to form teams at, so each team would be unable to hear or see the participants at other tables. The connection details were emailed to participants, and they were able to choose their own avatars and move around the space.

We also made a Zoom room available at the same time as the quiz as an alternative option.

End of event Meetup

Finally, to close out the event, a participant facilitated a virtual meetup in Mozilla Hubs, where participants took a virtual train ride to a venue where we celebrated all that we achieved during the workshop and Hack Day. The facilitator designed the space and added various objects that participants could interact with. The connection details were shared with participants via Slack and Zoom, and they were able to choose their own avatars and move around the space.

We also made a Zoom room available at the same time as an alternative option.

CW21 Publicity

Before the event

To promote the event we selected a freely-usable image of a fairground from Unsplash, a play on the theme of FAIR Research Software:

This image would be the default promotional image for CW21-related blog posts and news items on the Institute website, and default graphic for social media posts (Twitter and LinkedIn). We followed the syntax for the social media hashtag from previous years to remain consistent: #CollabW21. You can find our publicity content plan below, which consists of news items published to the Institute website for announcements such as keynote speakers, the call for submissions and financial assistance. These news items were then promoted via Twitter and LinkedIn, as well as emailed directly to the target CW21 Audiences. For context CW21 took place from 30 March - 1 April.

Publicity content plan

Round	Week of	Content	Published links
1	2020-10-12	News item: Collaborations Workshop 2021 (CW21) - Save the date!	Website, Twitter, LinkedIn
2	2020-11-16	News item: Registration open for Collaborations Workshop 2021	Website, Twitter, LinkedIn
3	2020-11-23	News item: Dr Michelle Barker to open Collaborations Workshop ‘21 with keynote on FAIR Research Software	Website, Twitter, LinkedIn
4	2020-11-30	News item: Call for submissions to Collaborations Workshop 2021	Website, Twitter, LinkedIn
5	2020-12-07	News item: Financial assistance available for Collaborations Workshop 2021	Website, Twitter, LinkedIn
6	2021-01-11	News item: Deadline for CW21 mini-workshop and social activity proposals: 31 January	Website, Twitter, LinkedIn
7	2021-02-01	News item: Call for lightning talks at Collaborations Workshop 2021Deadline for CW21 mini-workshop and social activity proposals: 31 January	Website, Twitter, LinkedIn
8	2021-02-08	News item: Dr Chonnettia Jones to deliver keynote on diversity and inclusion at Collaborations Workshop 2021	Website, Twitter, LinkedIn
9	2021-02-15	News item: Announcing the panel on diversity and inclusion at Collaborations Workshop 2021	Website, Twitter, LinkedIn
10	2021-02-22	News item: Announcing the mini-workshops and demo sessions at Collaborations Workshop 2021	Website, Twitter, LinkedIn
11	2021-03-15	News item: RemotelyGreen seeking beta testers for latest version [platform sponsor announcement]	Website, Twitter, LinkedIn
12	2021-03-22	News item: The Wellcome Trust to sponsor Collaborations Workshop 2021	Website, Twitter, LinkedIn

During the event

All sessions were recorded to be made available after the event was over. The keynote presentations and panel were live streamed.

During the event, the Institute Communications Officer live-tweeted using the #CollabW21 hashtag, in particular to engage community members who were unable to participate in the full programme to engage with the live streamed keynote presentations and panel. The Communications Officer also conducted interviews with participants to capture their experiences to promote future editions of CW.

Participants were also encouraged to share about their experiences during the event using the hashtag. During the welcome session at the start of each day, we reminded participants not to publicly share any links to the Zoom rooms or collaborative note documents. We also added a disclaimer to the header of each document reminding participants not to share links to the documents until after the event had ended. This was to protect our participants and event from “Zoom-bombers” or other intruders with malicious intent. Finally, we included a Social Media Policy within the event Participation Guidelines so that participants knew exactly what could and could not be shared on social media. It included the following:

For keynote presentations and mini-workshop sessions, it is assumed by default that information and images can be published on social media unless the presenter expressly states otherwise.

For collaborative sessions (such as Discussion groups, Collaborative Ideas or Hack Day teams), any publication on social media must first be checked with the rest of the group/team.

Any conversations in workshop communication channels (such as the Slack workspace or Zoom chat) must not be published on social media without the express permission of those involved.

Any conversations that take place within the social programme must not be published on social media without the express permission of those involved.

After the event

You can find our publicity content plan for after the event below, which consists of a blog post summarising the highlights from the event and a news item announcing the availability of the session recordings published to the Institute website. These were then promoted via Twitter and LinkedIn.

Publicity content plan

Round	Week of	Content	Published links
13	2021-05-24	Blog post: Highlights from Collaborations Workshop 2021	Website, Twitter, LinkedIn
14	021-07-12	News item: Videos from Collaborations Workshop 2021 now online	Website, Twitter, LinkedIn

In addition to the above content which we had to create, we also expected to publish a number of speed blog posts from the Discussion session groups to the Institute website. To make them easy to find, we have tagged them as CW21 speed blog posts and have also promoted them via social media, tagging the authors who provided handles.

CW21 Sponsorship

We adapted the sponsorship packages from CW20 to add more value to sponsors in a virtual event and summarised the packages on the CW21 website. We kept the tiered packages of Silver, Gold and Platinum, with increasing benefits respectively. Benefits we added include:

• Place on CW21 Steering Committee
• Increased number of free registrations
• Slide shown during breaks
• Dedicated breakout room for people to speak to you during the breaks at the event
• Lightning talk slot
• Video interview with Sponsor shared via social media (Platinum)

We invited partners that we have worked with previously, and also new partners related to the event themes and audience. We also added a sign-up form on the website for interested sponsors to contact us via.

We were delighted that F1000 Research, Figshare and eLife joined us again as Silver sponsors, and Wellcome joined us as a Platinum partner. Representatives from the sponsoring organisations engaged with the programme through our Steering Committee, lightning talks, and Hack Day judging. Figshare also provided a CW conference portal to capture and generate DOIs for the event outputs.

Sponsorship allowed us to procure additional platforms to enhance our event (such as a premium GatherTown subscription to facilitate the quiz within our social programme, Sli.do professional for audience Q&A and engagement), pay for speaker fees and gifts, and offer more financial assistance to participants.

A lesson learned with regards to sponsorship, is to include in the sponsorship packages whether or not they are inclusive of VAT to help manage expectations. We also learned that timelines for delivering benefits should be included on sponsorship package items to mitigate against the risk of a lot of benefits needing to be implemented at the last minute (e.g. due to partnerships secured close to the event date).

CW21 Venue

Zoom was used as the main workshop venue, please see CW21 Infrastructure for an extensive list of the platforms used to organise and run CW21.

• Capacity

  • The main plenary Zoom room’s capacity was 300, sufficient to fit the limit of 150 registrants.
  • Five additional Zoom accounts with a minimum capacity of 100 were used for the parallel sessions.
  • The facility to easily create and manage breakout rooms supported the event requirements.
  • Additional platforms with sufficient capacity to support virtual networking were used as part of the social programme (RemotelyGreen, GatherTown and Mozilla Hubs).

• Areas that support additional event requirements

  • Zoom breakout rooms were created to facilitate the Icebreaker session, Discussion session, Collaborative Ideas session, Networking session, and Hack Day.
  • Zoom breakout rooms were additionally created for break spaces, quiet spaces, sponsor booths, video interviews, and side conversations/meetings.

• Location

  • Having a virtual meeting allowed people to attend from locations around the world.

• Infrastructure

  • See CW21 Infrastructure.

• Accessibility

  • Having a virtual meeting allowed for greater accessibility.
  • Otter.ai integration with Zoom allowed for live transcription and captioning.

• Environmental sustainability considerations

  • Having a virtual meeting allowed for greater environmental sustainability due to the lack of travel, printed materials, and catering.

CW21 Event Roles

We had similar event roles as at CW20. Again we ensured that we had at least two people assigned for each role/duty as contingency in case someone had to drop out of the event for any reason. CW is the premier event of the Institute, so all staff were expected to attend and able to fill most roles. Institute Fellows, collaborators and CW21 Sponsors filled remaining roles on the Code of Conduct Committee and Hack Day judging committee.

We reused documentation from CW20 to make sure that everyone knew their roles and responsibilities: we created a duties roster, wrote instructions for Zoom Hosts and Co-hosts, created a spreadsheet of the various roles and Zoom room links for the parallel sessions, and provided an index of links to point to all these documents and to help people find anything else related to CW21 that they would need (links to each day’s collaborative notes document, Zoom rooms, communication channels, session information, etc.).

We organised test calls for CW21 helpers and Zoom Hosts to make sure all the necessary features were enabled for each account (breakout rooms, co-hosts, non-verbal feedback, automatic recording, live transcription and captioning via Otter.ia, etc.) and to let the Hosts gain experience using these features.

See the CW21 Timeline for a breakdown of the various tasks and when they took place.

Before the event

• Event Lead/Chair

  • Chair Steering Committee meetings
  • Chair Organising Committee meetings
  • Invite and coordinate with speakers
  • Process and caption pre-recorded keynote presentations
  • Invite and coordinate with sponsors
  • Deliver sponsorship benefits
  • Prepare publicity and content
  • Prepare, procure and test event infrastructure
  • Facilitate registration
  • Facilitate calls for submissions (mini-workshops and demo sessions, lightning talks)
  • Prepare for and mitigate risk of disruptions to the event
  • Procure prizes for Collaborative Ideas and Hack Day winners
  • Create duties roster and assign roles and tasks
  • Draft the “Before the event” Publicity

• Organising Committee members

  • Attend Organising Committee meetings
  • Perform agreed Organising Committee actions
  • Help with event infrastructure

• Steering Committee members

  • Attend Steering Committee meetings
  • Perform Steering Organising Committee actions

• Keynote speakers

  • Pre-record presentation and send to Event Lead

During the event

• Event Lead/Chair

  • Communicate connection details to participants via email and Slack

  • Act as Zoom Host for the main Zoom room

  • Create and manage breakout rooms

  • Synchronise participants via the collaborative notes documents

  • Reminding participants

    • To sign up for Discussion session topic
    • To vote for Collaborative Ideas
    • To register teams and projects for Hack Day
    • Of social programme activities they can take part in

  • Choreograph live streaming of keynotes and panel from Zoom to YouTube and Otter.ai via Restream

• Code of Conduct Committee members

  • Handle and resolve any reports of misconduct

• Slack monitors

  • Monitor the #help-desk and #general Slack channels and answer/relay any questions that come up

• Social media monitors

  • Tweet, retweet and engage from the @SoftwareSaved account
  • Monitor #CollabW21 Tweets to ensure that no links are being shared
  • Answer/relay any questions that come up

• Interviewer

  • Interview participants for video outputs from event and future promotion

• Zoom participant managers

  • Admit people from the waiting room
  • Mute (or unmute) participants as needed
  • Keep an eye on raised hands
  • Help with breakout room creation as needed

• Session Chairs

  • Introduce the speaker(s) or session and facilitate the session including Q&A

• Mini-workshop and demo session Zoom Hosts

  • Enable the waiting room and admit participants
  • Assign Zoom Co-hosts
  • Ensure the session is recording
  • Ensure the live transcription and captioning is working
  • Assist the Mini-workshop and demo session Facilitator as needed

• Mini-workshop and demo session Zoom Co-hosts

  • If the Zoom Host is not present (e.g. unable to connect or attend), the Zoom Co-host is the backup

  • Help with Zoom participant management:

    • Admit people from the waiting room
    • Mute participants as needed
    • Help with breakout room creation as needed
    • Assist the Zoom Host as needed

• Mini-workshop and demo session Facilitators

  • Deliver session
  • Use the generated collaborative notes document for their session as needed to synchronise participants

• Keynote speakers

  • Deliver Q&A

• Panellists

  • Deliver panel

• Lightning talks speakers

  • Deliver lightning talk

• Hack Day Judges

  • Visit teams
  • Judge and score the projects
  • Award prizes

After the event

• Event Lead/Chair

  • Close down the Event Project Stage

    • Send out feedback form to participants
    • Send prizes to Collaborative Ideas and Hack Day winners
    • Gather and prepare recordings for processing
    • Coordinate with Discussion session groups about their speed blogs
    • Follow up with Finance on sponsorship invoices
    • Draft the “After the event” Publicity
    • Write this In Practice on CW21

• Organising Committee members

  • Process the session recordings (slice video recordings into individual sessions/speakers, edit captions, publish on YouTube)
  • Proofread, edit, publish and promote Discussion session speed blogs
  • Proofread, edit, publish and promote any “After the event” Publicity

CW21 Event Safety

Ahead of the event, we attended a Zoom safety drill hosted by the CSCCE to test out the various security features available on Zoom, and to role play what it would be like to have to report and evict a bad actor and then deal with the fallout. The security features within Zoom make it easy to stop everything in order to eject a bad actor, but this can be quite jarring and disruptive for participants (and traumatising to those targeted by the bad actor) so ensuring a safe environment for participants from the start is of utmost importance. A Code of Conduct mandates standards of behaviour and interaction expected from organisers and participants.

Code of Conduct

The Code of Conduct was provided as a set of documents:

• The Code of Conduct itself: CW21 Code of Conduct
• How to report a violation: the CW21 Attendee Procedure for Reporting Harassment
• How the Code of Conduct committee will handle harassment reports: the CW21 Staff Procedure For Handling Reported Harassment

All of these materials were made available within the CW21 Participation Guidelines which participants had to have read and agreed to abide by at registration, and were reminded of throughout the event.

CW21 Risk Management

For CW21, we looked at the following risk aspects:

• The main stakeholder and funder (the Software Sustainability Institute)
• Other stakeholders (sponsors, keynote speakers, attendees, session chairs, organising committee members, volunteers/helpers at the event)
• Event agenda
• Venue
• Communication (with various stakeholders)
• Policies, guidelines and documentation

Unlike for CW20, the global pandemic was at the forefront of our risk assessment and mitigation plan for CW21. We decided from the beginning that CW21 would take place online for two main reasons:

1. It was clear that the pandemic would be ongoing and there would still be hesitancy from both organisers and participants towards traveling to an in-person event.
2. It was an opportunity to improve and build on what we achieved running CW20 Online.

It turned out that there were still many restrictions around international travel and in-person events at the time of CW21, so we successfully mitigated the risk of having to cancel another in-person event.

The main stakeholder

The risks that we identified and needed to mitigate relating to the main stakeholder - in this case the Software Sustainability Institute - were:

• Event goals and objectives not aligned with the main stakeholder goals
• Damage to the main stakeholder’s reputation

As we demonstrated through CW20, the event goals and objectives can be achieved through a virtual event. To ensure that the CW21 Goals and Objectives aligned with the main stakeholder goals, the Director was consulted during the CW21 Feasibility Stage, and both the Director and Research Director were invited to participate on the CW21 Steering Committee.

As we demonstrated through CW20, the main stakeholder’s reputation would not be damaged by running a virtual event. In fact, by running CW20 virtually at the start of the pandemic, the Institute was internationally recognised and regularly consulted on virtual event organisation and community building. Additionally, 70% of CW20 participants who gave feedback stated that “CW20 is the best [online] event I have attended”.

Feedback from CW20 participants on how the event compared with other online events

Therefore, due to the attention and care that we put into improving and building on what we achieved during CW20 and mitigating the risk of damaging the Institute’s reputation, we had similar feedback after CW21 when compared with the past year of virtual events.

Feedback from CW21 participants on how the event compared with other online events

Other stakeholders

Attendees

The risks identified regarding the attendees include:

• Not enough attendees or many dropping out right before the event
• Too many attendees

The strategy we took to mitigate these risks included:

• Keep a registration fee in place for a virtual event to minimise no-shows
• Limit the number of registrations to 150 so that we could manage the interactive sessions and have a more intimate event

Within the CW20 feedback form, we asked participants how they would prefer us to organise CW in the future (in-person or online). Half of respondents said that they didn’t mind how it was run, and 33% said that they prefer an in-person event.

Feedback from CW20 participants on how to organise the event in the future

However, as the effects of the pandemic were ongoing, we decided that CW21 would take place online. As the vaccination programmes advance and things begin to open up, there is pressure and expectation for hybrid events. We asked CW21 on the feedback form to rank how they prefer future CWs to be organised: in-person, online, hybrid or don’t mind, the results of which are displayed below:

Feedback from CW21 participants on how to organise the event in the future

Opinions were more or less evenly split between the different options, and we believe this may be due to ongoing uncertainty about the pandemic.

Keynote speakers

Risks that we identified from running CW20 online included sickness or being unable to present due to caring or other responsibilities imposed due to the pandemic restrictions. Connection or technical issues could also arise. The strategy we took to mitigate these risks was to request the keynote speakers to pre-record their presentations. We provided a speaker guide with advice for recording their talk, and our full support with editing and captioning the videos.

This strategy had the added benefit that it allowed us to have keynote speakers from regions with time zones incompatible with the times of the event - the speakers did not have to present outside of their normal working hours.

Session chairs

The risks identified regarding the session chairs included connection or other technical issues. We also identified sickness or being unable to present due to caring or other responsibilities as risks from running CW20 online. The strategy we took to mitigate these risks was to have backups for every CW21 Event Role .

Organising Committee members

The risks identified regarding the Organising Committee members and the strategy we took to mitigate these risks were the same as during CW20. In particular, the largest risk was members being unable to help run the event on the day due to connection issues, schedules, sickness or being unable to present due to caring or other responsibilities. Our mitigation strategy to have backups for every CW21 Event Role still applies to these scenarios.

Volunteers/helpers at the event

The risks identified regarding volunteers and helpers at the event and the strategy we took to mitigate these risks were the same as during CW20.

Event agenda

The risks identified regarding the agenda included:

• The social activities are not attended or unsuccessful
• The days are too long for an online event leading to Zoom fatigue
• Some sessions don’t translate well to an online setting
• Lose nascent quality of unconference event
• Technical issues
• Bus factor

The strategy we took to mitigate these risks included:

• Have a variety of sessions to keep participants engaged
• Schedule sufficient breaks throughout the day
• Provide time for networking within the main programme to encourage networking between participants
• Allowing participants to choose their own breakout rooms and have edit access to the collaborative documents improved nascent quality of virtual unconference
• Documenting procedure for running a session in case the Session Chair is unable to facilitate the session

Venue

One risk to using Zoom was that participants may be bored of Zoom and looking for new and exciting alternative platforms. However, we had the most experience with it and it had the best performance, scalability and functionality out of the many platforms we tested. We attended many of the CSCCE Tools Trials which tested and compared features from a variety of virtual conferencing platforms, including for QuiqoChat, Etherpad with video and Wonder, and found that Zoom still best met our needs. The ability to create and manage breakout rooms is essential for our interactive sessions such as the Discussion session, Collaborative Ideas session and Hack Day. The added functionality in Zoom allowing participants to self-select their own breakout rooms was a game changer (we did not have this for CW20) and provided more autonomy for participants. To provide variety to participants, we used alternative platforms for the social programme activities.

Communication

The risks identified regarding communication included:

• Participants confused about the agenda or not getting information on the right channels and at the appropriate time/frequency
• Too many or not enough pathways to engagement

The strategy we took to mitigate these risks included:

• Emails to registrants communicating information regarding connection details, programme, and general guidance for getting the most out of the virtual event
• Documentation for the delivery team (comprised of Session Chairs, the Organising Committee and volunteers/helpers), such as Zoom instructions for hosts, a duties roster, etc. to make sure that everyone was on the same page and knew their responsibilities
• Documented all relevant instructions, links and other information in the participant-facing collaborative notes document to keep everyone synchronised
• Slack workspace for the event that would persist outside of the Zoom meeting for sharing information and participant engagement
• Use of Sli.do to provide an interactive element for participants and an anonymous way of asking questions

Policies, guidelines and documentation

The risks identified regarding policies, guidelines and documentation included:

• Lack of clearly stated policies and guidelines suitable for an online event
• Data privacy issues related to using Zoom and risk of “Zoom-bombing”
• Data privacy issues related to which documents could be shared and how
• Risk of connection details (such as the links to Zoom rooms, collaborative notes documents, transcriptions, and Slack channel) being shared publicly
• Issues relating to Intellectual Property

The strategy we took to mitigate these risks included:

• Develop a set of Participation Guidelines for the online event comprised of a Code of Conduct, Privacy Policy, Social Media Policy, Intellectual Property Policy and Change Log (to note any updates to the guidelines leading up to the event)

  • A lesson learned from CW20 was to update the Privacy Policy and add a license to the collaborative documents to enable easier sharing of outputs after the event

• Disclaimer and license added to all collaborative documents generated so that participants would know how they could be used going forward and decide what information they were willing to share

• Remind participants about the Participation Guidelines at the start of each day

• Remind participants not to publicly share any of the Zoom room links or collaborative documents which contain links to the Zoom rooms and Slack channel

CW21 Outputs and Outcomes

Outputs

• Recordings of the sessions shared via the SSI Youtube channel

  • Full event playlist
  • Keynote presentations live stream
  • Panel live stream
  • Keynotes and panel playlist
  • Lightning talks playlist
  • Mini-workshop and demo sessions playlist
  • Hack Day demos playlist

• Blog posts from the discussion session

  1. Implementing FAIR for research software: attitudes, advantages and challenges (part one)
  2. Implementing FAIR for research software: attitudes, advantages and challenges (part two)
  3. What are the formats, tools and techniques for harvesting metadata from software repositories?
  4. A path to the light: stopping ‘secret’ software, managing maintenance and evidencing impact
  5. Top tips for teaching what’s REALLY important
  6. Nudging towards a better default for open source project ownership
  7. Evolving the office work culture
  8. How do we improve data management in machine learning?
  9. Coding Clubs For Research Software Communities: Questions to Consider (Part One)
  10. Coding Clubs For Research Software Communities: Questions to Consider (Part Two)
  11. Six tips for building communities of practice
  12. How do you help build intermediate software engineering skills and help people go beyond the basics?
  13. Talk to each other!

• Collaborative Ideas documents

  1. User centred design principles for research apps / citizen science
  2. Research Software Sustainability Concordat
  3. Dugnads for Research
  4. Escape room: Translating between RSEs and Arts & Humanities Researchers (Part 1) (3rd place winners)
  5. Coding Confessions
  6. Acknowledging maintenance and software retirement: Software End of Project Plans (1st place winners)
  7. DEPTH: Developer error/problem treasure hunt!
  8. Improving data visualization literacy
  9. Using Raspberry Pis to deliver Carpentries training in remote locations (tied 2nd place winners)
  10. Software citation videos (to understand why and how to make your software citable)
  11. The Adventurous Architect - A Blueprinter for Research Software Skills
  12. ConnECT ProjECT - an Exciting Collaboration Tool for discovering project similarities!
  13. Skills wheel for RSEs
  14. Bursting the Bubble: Teaching PIs the value of good code (tied 2nd place winners)
  15. Open Source Covid Analysis of References (OSCAR)

• Mini-workshop and demo session documents

  • 1.1 Interactive Introduction to the FAIR Research Software discussion
  • 1.2 The RSE landscape: Central, service, embedded, academic - what is your RSE type, and how do you want to develop it?
  • 1.3 README tips to make your project more approachable
  • 1.4 How FAIR is your research software?
  • 1.5 (Do not) make it new: On Reusing Research Software and Tools in Digital Humanities Scholarship
  • 2.1 Tips and traps on the road to FAIR software principles
  • 2.2 Good Practices for Designing Software Development Projects (The Turing Way)
  • 2.3 Promoting code review for research software: feedback from the Oxford Code Review Network
  • 2.4 PresQT – Services to Improve Re-use and FAIRness of Research Data and Software
  • 2.5 Opening Closed Data - Exploring new models for stewardship of sensitive data

• Hack Day projects

  1. HowDescribedIs
     • GitHub action
     • Test GitHub repository
     • Automatically generated pull request
  2. Coding Confessions (2nd place winners)
     • Website
  3. Research Dugnads (tied 3rd place winners)
     • Website
  4. Escape room
     • Website
     • GitHub repository
     • Presentation
  5. CarpenPi (1st place winners)
     • Organisation page on GitHub
     • Workshop leader training material
     • Project wiki for project documentation
     • Presentation
  6. Habeas Corpus (tied 3rd place winners)
     • GitHub repository
  7. Software Twilight
     • GitHub repository (spawn a Binder session to see it in action)
     • Collaborative notes
  8. Credit All
  9. Exploring past CW ideas (special mention)
     • Website
     • GitHub repository
     • Presentation

• All slides and other outputs shared via the Figshare conference portal

  • Keynote presentation slides
  • Panel presentation slides
  • Lightning talk slides (x20)
  • Mini-workshop and demo session presentation slides or outputs (x5)

• A highlights blog post on the SSI website

• Collaborative notes documents and other unconference infrastructure resources shared

  • CW21 Day 1 Notes
  • CW21 Day 2 Notes
  • CW21 Hack Day Notes

• Feedback into Event Organisation Guide (this In Practice on CW21)

• Any sponsor outputs (e.g. Tweets, videos, etc.)

  • News item announcing Platinum Sponsor
  • Video announcing Platinum Sponsor

Outcomes

• Facilitation of new collaborations between participants

  • Feedback from CW21 participants included that 28.6% of respondants met more than 15 new people at the workshop, 68.6% of respondants started 1-2 collaborations based on conversations at the workshop, and 80% of respondants intend to carry on working on the idea they developed at the CW21 Hack Day.

Feedback from CW21 participants on how many new people they met at CW21

Feedback from CW21 participants on how many collaborations did they start, or intend to start, based on conversations from CW21

Feedback from CW21 participants on whether they intend to carry on working on the idea they developed at the CW21 Hack Day

• Increased accessibility and inclusion

  • Feedback on the use of live transcription and captioning from participants included: “I am an non-native english speaker; reading helps with understanding”, “I have a slight hearing impairment which sometime makes it difficult for me to follow a conversation. I like being able to use the transcription.” and “It is super helpful to be able to go back to a sentence if I misheard anything”.

Feedback from CW21 participants on whether it was helpful / more accessible to have live transcription and captioning during the workshop

• Community members who are unable to attend the event are still able to engage with the discussion

  • A Tweet from a community member who was able to engage via the live streamed sessions

• Impact on virtual events run by others through influencing and informing how they run their virtual events and reuse of infrastructure resources

  • After experiencing the use of Gather.Town at CW21, SSI Fellow Emma Rand used the platform to facilitate Software Sustainability training for Bioscience Postgraduate students

• Ease of running a virtual CW again in the future

  • We are able to run CW22 following the same process detailed within this In Practice on CW21 and reuse organisational and infrastructure resources with minor modifications

CW21 Closing Down the Event Project Stage

At the end of the event

We made participants aware that we would be gathering feedback, following up with them about speed blogs and their prizes at the end of the event. We thanked the sponsors, volunteers, speakers and workshop facilitators for their contributions towards delivering the event, and we also thanked the participants for their engagement - without which the unconference would not have been a success.

After the event

We emailed participants with a form to collect feedback within a week of the event.

In the months after the event, we followed up on the speed blogs that were produced as part of the Discussion session. We published them along with reflection blog posts, collected and shared the presentation slides, and processed and published the session recordings. You can see the details of the CW21 outputs in the CW21 Outputs and Outcomes and the timeline of when we made them available in the CW21 Timeline.

When are you done?

All of the outputs that we planned to produce are now available (locations of which are mentioned in the section above) and the event is complete.

About

Authors

• Shoaib Sufi (lead/maintainer) (ORCID)
• Selina Aragon (ORCID)
• Aleksandra Nenadic (ORCID)
• Rachael Ainsworth (ORCID)
• Neil Chue Hong (ORCID)

Other contributors:

• Mario Antonioletti (ORCID)
• Raniere Silva (ORCID)
• Clem Hadfield
• Robin Wilson (ORCID)
• Malvika Sharan (ORCID)
• Jacalyn Laird

Licence

The Event Organisation Guide (EOG) material is available under the Creative Commons Attribution 4.0 Licence. Please see the human-readable summary of the CC BY 4.0 and the full legal text for further information.

Acknowledgements

The work carried out by the Software Sustainability Institute is supported by:

• EPSRC, AHRC, BBSRC, ESRC, MRC, NERC, STFC grant EP/S021779/1
• EPSRC, BBSRC and ESRC grant EP/N006410/1
• EPSRC grant EP/H043160/1

Cite

The Software Sustainability Institute Event Organisation Guide (SSI-EOG) can be cited using the DOI 10.5281/zenodo.3970897.