Add methods in dslite.server

Support Developer support
1

Dear all,

I was developing a function in DSLite.

1, To define a DSLite server, it seems every functions used in server have to be decleared using commands like dslite.server$assignMethod(s). If I want to add all finctions of a package i.e. “ase”, is there any fast way to do?

2, could you please check the codes below, why I get the error for the code ds.colnames(“df”)

datashield.errors()
$server1
[1] "DataSHIELD configuration does not allow expression: exists\nSupported function calls are: class"

$server2
[1] "DataSHIELD configuration does not allow expression: exists\nSupported function calls are: class"

$server3
[1] "DataSHIELD configuration does not allow expression: exists\nSupported function calls are: class"

See below:

library(resourcer)
library(DSLite)
library(DSOpal)
library(opalr)

cnsim1 <- resourcer::newResource("CNSIM1", "https://github.com/datashield/DSLite/raw/master/data/CNSIM1.rda", format = "data.frame")
cnsim2 <- resourcer::newResource("CNSIM2", "https://github.com/datashield/DSLite/raw/master/data/CNSIM2.rda", format = "data.frame")
cnsim3 <- resourcer::newResource("CNSIM3", "https://github.com/datashield/DSLite/raw/master/data/CNSIM3.rda", format = "data.frame")
dslite.server <- newDSLiteServer(resources = list(cnsim1 = cnsim1, cnsim2 = cnsim2, cnsim3 = cnsim3))
dslite.server$resourceNames()
dslite.server$config()
dslite.server$assignMethod("as.df", "base::as.data.frame")
dslite.server$aggregateMethod("class", "base::class")

builder <- DSI::newDSLoginBuilder()
builder$append(server = "server1", url = "dslite.server", resource = "cnsim1", driver = "DSLiteDriver")
builder$append(server = "server2", url = "dslite.server", resource = "cnsim2", driver = "DSLiteDriver")
builder$append(server = "server3", url = "dslite.server", resource = "cnsim3", driver = "DSLiteDriver")
logindata.dslite.cnsim <- builder$build()

conns <- datashield.login(logindata.dslite.cnsim, assign=T)
datashield.assign.expr(conns, "df", quote(as.df(D)))

library(dsBaseClient)
ds.colnames("df")
2

Hi,

  1. There is no shortcut function for adding ALL functions of a standard R package. The DS compliant functions needs to be declared explicitly. I do not know the “ase” package, if you are the author of this package, you can add a DS configuration file in it, that will be discovered at runtime when building a DSLiteServer object. See examples of this kind of file: resourcer/inst/DATASHIELD or dsOmics/inst/DATASHIELD.

  2. The error reports that the function exists() is not defined in the DS configuration. This is probably because you do not have installed the dsBase package: with DSLite, both client-side and server-side packages needs to be installed.

I have updated your script with some comments:

library(resourcer)
library(DSLite)
# note: no need to load DSOpal because no connection to Opal is done
# note: no need to load DSI because it is a dependency of DSLite

cnsim1 <- resourcer::newResource("CNSIM1", "https://github.com/datashield/DSLite/raw/master/data/CNSIM1.rda", format = "data.frame")
cnsim2 <- resourcer::newResource("CNSIM2", "https://github.com/datashield/DSLite/raw/master/data/CNSIM2.rda", format = "data.frame")
cnsim3 <- resourcer::newResource("CNSIM3", "https://github.com/datashield/DSLite/raw/master/data/CNSIM3.rda", format = "data.frame")
dslite.server <- newDSLiteServer(resources = list(cnsim1 = cnsim1, cnsim2 = cnsim2, cnsim3 = cnsim3))
dslite.server$resourceNames()
# note: this will list the DS configuration (discovered at runtime), i.e. aggregation, assign methods and R options
# you should expect to see config from dsBase (if the dsBase package is installed) and from resourcer
dslite.server$config()
# note: this one is redundant with as.resource.data.frame defined by resourcer
dslite.server$assignMethod("as.df", "base::as.data.frame")
# note: this one is redundant with classDS defined by dsBase
dslite.server$aggregateMethod("class", "base::class")

builder <- DSI::newDSLoginBuilder()
builder$append(server = "server1", url = "dslite.server", resource = "cnsim1", driver = "DSLiteDriver")
builder$append(server = "server2", url = "dslite.server", resource = "cnsim2", driver = "DSLiteDriver")
builder$append(server = "server3", url = "dslite.server", resource = "cnsim3", driver = "DSLiteDriver")
logindata.dslite.cnsim <- builder$build()

conns <- datashield.login(logindata.dslite.cnsim, assign=T)
datashield.assign.expr(conns, "df", quote(as.df(D)))

library(dsBaseClient)
ds.colnames("df")
# note: result is NULL when dsBase package is installed
datashield.errors()

# note: same result, using resourcer's function
datashield.assign.expr(conns, "df", quote(as.resource.data.frame(D)))
ds.colnames("df")

datashield.logout(conns)

Cheers
Yannick

3

Hi @yannick, I just read your very informative comment and wondered whether the AggregateMethods are no longer defined in the DESCRIPTION file but in /inst/DataSHIELD? Is this a breaking change in Opal 3.0? Or are both variants possible?

4

Hi Stefan,

Yes, both variant are possible (see dsBase). It’s just that it is not allowed by the CRAN policy to have these heterodox entries in the DESCRIPTION file. In order to get the resourcer package accepted in the official CRAN, the alternate solution was to have the same declarative syntax in a separate file. As the R packaging rule is that any files in the inst/ directory in the package’s source code will end up in the root of the package’s installation directory, Opal (3.0) and DSLite are looking for DS config entries either in the DESCRIPTION file or in the DATASHIELD file.

So if you want to stick with the CRAN policy, using a DATASHIELD file is recommended.

Best
Yannick

1 Like
5

Thanks for this clarification!

6

Dear Yannickm,

Thanks a lot, your comments are quite useful. I would see we can add the functions to DS server from the established R package. Is it possible to add the functions to DS server from the .R files directly in the developing stage?

Regards, Hank

7

Yes, you can do it manually in the Opal web interface (go to Administration > DataSHIELD and then edit aggregation/assignment methods) or from R using the dsadmin.set_method. In both cases, it requires administration privilegies.

Yannick

8

Great! thanks a lot

Regards, Hank