DataSHIELD roadmap development

Support Developer support
1

Hi all,

We have been putting together a draft roadmap of the priorities at the Newcastle site over the last few weeks. We now have a proposed plan - see below. The headlines for this are:

  • Major release (v5.0) by the time of the conference in September.
  • Testing framework up and running by September.
  • Major release (v6.0) soon after the conference to switch to DSI as main interface.

This roadmap covers the main work packages and milestones from various grants that we are involved in. Please have a read and see if it aligns with your deliverables etc and let us know any problems or concerns.

https://data2knowledge.atlassian.net/wiki/spaces/DSDEV/pages/658505729/Road+map

Olly

2

Hi Olly, here are some comments from me:

  • How to keep this roadmap alive and make sure it becomes the go-to place where people can look to find out what is happening? Perhaps a regular status at the EUCAN WP3/5 call can drive this? For example it would be great when the process becomes more established to see future releases plotted out on a timeline, with functionality assigned to each release. I can see this is not possible at the moment while things are being sorted out…
  • Once you have the development process confirmed internally, how to roll it out for ‘external’ developers to contribute their own functions (EUCAN T3.7) to moved development burden towards those who want the new functions
  • Something about developing ‘how-to’ guides for actually doing the analysis (so-called bioinformatics toolbox referenced in EUCAN WP5)
  • Test data set with clustered data for testing mixed model functions, merge functions etc.
  • As well as text processing and GUI, there are other data types such as genomic / microbiome data that are referenced in EUCAN
3

Hi Tom,

thanks for the comments!

I’ve found the scope of this roadmap difficult to pin down, there are currently three funded DataSHIELD projects led out of Newcastle and I know of three more big ones being applied for at the minute. Added to that are the other DataSHIELD projects which we have no involvement in at all. With this in mind I am approaching this as a non-project specific DataSHIELD plan where the lead sits at Newcastle, with the aim of letting others know what we are doing, and not as an all encompassing plan that has everything DataSHIELD in it. I’d welcome others’ thoughts on this as to if it is useful or not! I’d potentially be open to delegating some editorial rights so others could add to this.

  • I agree about the need for keeping the roadmap alive. A regular (3/6 monthly?) slot in EUCAN calls makes a lot of sense.
  • It’s all about the testing at the minute. Once we have the framework in place the plan is to write tutorials to show how to write tests on new functions etc. Then we need a policy discussion (aiming for the September workshop) to decide what level of community acceptance is needed for us to accept a new function into the core repos. e.g. what tests are required, do we need code reviews etc.
  • It’d be good to understand the test data sets you need and to make them early on.

Cheers!

4

Would it be useful to specify some milestones on the way to the September’s 5.0 Release? We could have milestone M1 (end-May), M2, (end-June) and M3 (end-July), and corresponding Engineering Release: 5.0-M1, 5.0-M2 & 5.0-M3.

We would need to pin down the new functions to be implemented, what state they should be at each milestone: proof of concept, complete but untested, complete and tested, …

Stuart

5

Hi Olly,

I can completely see the challenge you have with this. I think that managing the development process and roadmap is a full time job in its own right, without doing any of the additional testing work etc. But it is vital that people can have some sort of insight about what is coming when, and, as the processes become more define, how they might contribute their own work.

I will have a think about the test data set, and perhaps ask if there is something available from all the work LifeCycle have been doing that would fit the bill.

6

Hello,

Just a quick message to say that in RECAP we have already found it useful to look at the roadmap - mostly so that end users know what functionality might be available and when. I completely understand the difficulties in keeping information synced in different places, but @olly 's point about other projects outside of EUCAN also benefitting from awareness is spot on.

Cheers,

Andrei

7

Thanks Olly,

We are also happy to have this now. I would like to discuss and see a bit more detail of the ‘waiting list’ of features.

Because it is now very course grained features such as ‘UI’ but most of our users have more priority in ‘specific methods’.

In that light it would be great if RECAP can also create a list of their analysis needs. In LifeCycle I am also pushing for this. Thus we can inform and help priority setting in the roadmap.

Cheers

Morris