dsBetaTest support

Support Analyst Support
1

Hi,

I’m never quite sure whether I should be posting in this user forum or the dev one… Anyway, that is not the question!

I was discussing with Paul this morning about the new functions. They look great! However, I’m currently getting some errors, and not quite sure how to progress with that.

R version 3.3.3 (2017-03-06) -- "Another Canoe"
 R> ## LOAD PREREQUISITES                                                                                                                                                                                                         
 R> devtools::install_github("datashield/dsBetaTestClient")                                                                                                                                                                       
 Skipping install of 'dsBetaTestClient' from a github remote, the SHA1 (b1ef7dd6) has not changed since last install.
   Use `force = TRUE` to force installation
 R> library("dsBetaTestClient")                                                                                                                                                                                                   
 Loading required package: dsBaseClient
 Loading required package: opal
 Loading required package: RCurl
 Loading required package: bitops
 Loading required package: rjson
 Loading required package: mime
 Loading required package: fields
 Loading required package: spam
 Loading required package: dotCall64
 Loading required package: grid
 Spam version 2.1-4 (2018-04-12) is loaded.
 Type 'help( Spam)' or 'demo( spam)' for a short introduction
 and overview of this package.
 Help for individual functions is also obtained by adding the
 suffix '.spam' to the function name, e.g. 'help( chol.spam)'.

 Attaching package: ‘spam’

 The following objects are masked from ‘package:base’:

     backsolve, forwardsolve

 Loading required package: maps
 See www.image.ucar.edu/~nychka/Fields for
  a vignette and other supplements.
 Loading required package: dsStatsClient
 Loading required package: dsGraphicsClient
 Loading required package: dsModellingClient
 R> ## Set credentials....  
...
 R> ## login                                                                                                                                                                                                                      
 R> opals <- datashield.login(logins=logindata, assign=TRUE)                                                                                                                                                                      

 Logging into the collaborating servers

   No variables have been specified.
   All the variables in the opal table
   (the whole dataset) will be assigned to R!

 Assigning data:

...

 R> ds.listOpals.o()                                                                                                                                                                                                              
 Error: could not find function "ds.listOpals.o"
 R> ds.listClientsideFunctions.o()                                                                                                                                                                                                

 ### dsBaseClient functions                                                                                                                                                                                                       
  [1] "ds.asCharacter"    "ds.asFactor"       "ds.asList"         "ds.asMatrix"       "ds.asNumeric"
  [6] "ds.assign"         "ds.c"              "ds.cbind"          "ds.changeRefGroup" "ds.class"
 [11] "ds.colnames"       "ds.dataframe"      "ds.dim"            "ds.exists"         "ds.exp"
 [16] "ds.isNA"           "ds.isValid"        "ds.length"         "ds.levels"         "ds.list"
 [21] "ds.log"            "ds.ls"             "ds.mean"           "ds.meanByClass"    "ds.names"
 [26] "ds.numNA"          "ds.quantileMean"   "ds.recodeLevels"   "ds.replaceNA"      "ds.rowColCalc"
 [31] "ds.subset"         "ds.subsetByClass"  "ds.summary"        "ds.table1D"        "ds.table2D"
 [36] "ds.vectorCalc"


 ### dsGraphicsClient functions                                                                                                                                                                                                   
 [1] "ds.contourPlot" "ds.densityGrid" "ds.heatmapPlot" "ds.histogram"


 ### dsModellingClient functions                                                                                                                                                                                                  
 [1] "ds.gee"   "ds.glm"   "ds.lexis"


 ### dsStatsClient functions                                                                                                                                                                                                      
 [1] "ds.cor"     "ds.corTest" "ds.cov"     "ds.tTest"   "ds.var"


 ### userDefinedClient functions                                                                                                                                                                                                  
 character(0)
 R> 
 R> ds.listServersideFunctions.o()                                                                                                                                                                                                
 Error in ds.listServersideFunctions.o() :
   could not find function "dsadmin.get_methods"
 R>

QUESTION: why is ds.listServersideFunctions.o() not working? I have been working with the team in Porto at INESC-TEC who are running the (three) nodes that I am connecting to, and they’ve ensured that the latest version of dsBetaTest is installed on the nodes. So, what am I doing wrong? Thanks!

2

PS. obviously, I snipped some of the code. My local R instance is running via ESS on a Debian stable GNU/Linux system (but I don’t think that is related).

3

Hi,

dsadmin.get_methods is part of the opaladmin R package, which was not loaded (in the client-side R session).

Cheers Yannick

4

Others have experienced something similar. @PatRyserWelch has mentioned that you need to install all the DS packages (install.packages()) and load them all with library(), you will need to load all DS client libraries using library e.g. dsbaseclient, dsmodellingclient etc etc

My initial suggestion was that also maybe the nodes are not running the latest version from the master branch. Each node needs to ensure they install the dsBetaTest master branch - and not the release on obiba cran, You can install it by typing master in to the Git reference box in the Opal web interface for each node, and then clicking publish methods.

As a side note you can also check the list of functions available in a packge by using: >ls("package:dsBaseClient")

5

Hi,

The issue with ‘ds.listOpals.o()’ should be fixed in the latest version of ‘dsBetaTestClient’. Has the ‘ds.listServersideFunctions.o()’ issue been fixed by installing the ‘opaladmin’ package?

Stuart

6

Hi all!

@asm as we discussed, the nodes were running version 0.3.0 of dsBetaTest but they are now updated to version 0.4.2.

I was only able to install the latest version by running dsadmin.install_package(o, pkg='dsBetaTest', githubusername='datashield', ref='1a6c83c'). Opal gave me a 404 when specifying either master or the commit ref.

@swheater I just installed deBetaTestClient (also 0.4.2) and ds.listOpals.o worked fine. And ds.listServersideFunctions.o does work if I load opaladmin first.

Anyway, I think the issue that surprised Paul was that the dsBetaTestClient functions are not listed by ds.listClientsideFunctions.o.

And I just noticed that ds.listServersideFunctions.o does not list the dsBetaTest functions either.

7

Hello all, Thanks for all the responses! I hope to have a chance to look again at this later this week, so will confirm if working (or not) soon. @gfcg - yes, I think you are right about the issue that surprised Paul.

8

Hi,

For your information there are some very useful functions that can be used for inquiring the consistency of the datashield set up:

  • datashield.pkg_status List the datashield server side packages and their versions in each of the Opal servers.
  • datashield.method_status List the server side datashield functions and their availability in each of the Opal servers.
  • datashield.methods Get a detailed list of server side datashield functions (with the version of the package where they are defined) from each of the Opal servers.

Example:

# opals <- datashield.login(...)
datashield.pkg_status(opals)
datashield.method_status(opals)
datashield.methods(opals)
# datashield.logout(opals)

These functions are part of the original (i.e. opal specific) datashield interface and have been ported to the coming new datashield interface (data repository agnostic) in a backward compatible way.

Yannick

9

Hello again,

I’ve just had a chance to check and everything now works correctly :slight_smile: Thanks particularly for @gfcg for ensuring the nodes are up-to-date and also to @yannick for highlighting the need to include opaladmin (this was the one that got ds.listServersideFunctions.o() working) as well as pointing out the three commands that I didn’t know about looking at package status etc… I’m delighted to say that one of the nodes even had ds.mice on it!

Cheers,

Andrei