Dear all,
I came across various ways of installing DataSHIELD and wonder what the recommend is for newcomers:
Any advice?
Wilmar
Hi Wilmar,
Personally I have been transitioning from “Barebones” (automated by puppet) to docker compose based deployments (both Opal and Armadillo), also I am also experimenting with a CORAL deployment. Given I am mostly interesting in quality assurance I feel I need to cover both “Barebones” and “Docker” based deployments.
If a full deployment of all the Obiba (Opal, Mica, Agate, +) tools is your aim, then CORAL appears to be a good choice [my investigation is stalled due to a deployment issue].
Also it would be worth checking it Docker based deployment is supported by your IT department. We had to persuade our IT department to make some changes to their infrastructure to cope with Docker.
Stuart
Also there is Armadillo, but the documentation is under development.
For just getting to understand how DataSHIELD works I would recommend the Docker approach described here:
https://opaldoc.obiba.org/en/latest/admin/installation.html#docker-image-installation
Agreed with Stuart, Coral is nice for a full stack deployment, including studies and dataset documentation. A simple docker approach is enough for Datashield and is straightforward (that is how opal-demo has been deployed daily, for years).
Hi, I installed DataShield on srv3.stats-con.com and followed the instructions for the docker image installation [1] until: docker-compose -f docker-compose.yml up -d
What next?
Thanks, Wilmar
The docker-compose up command starts all the declared services (i.e. opal, rock and database(s)). You do not need to start Rock specifically, you do not need to access Rock (because opal does it in the docker’s internal network).
Again, you should not link a R server to several opal servers. There is no benefit to this and it can be confusing in terms of resource management.
Dear Yannick,
got it, if OPAL and ROCK are installed on the same server, ROCK is automatically registered and this ROCK installation should NOT be registered with other OPAL servers to avoid inconsistencies.
On srv3.stats-con.com, after opening the ports 8843 and 8880 I can now login using OPAL on https://srv3.stats-con.com:8843.
Login via srv3.stats-con.com:8880 shows “Authentication failed” when entering default credentials (as in docker yml) presumably because of a block of “Cross Site Request Forgery (CSRF)”
Thanks for your support,
Wilmar
Hi,
I am trying to login to the opal server with:
con.opal <-·opal.login(username·=·“administrator”,·password·=·“password”, url·=·“https://srv2.stats-con.com”)
R console Error: Error in curl::curl_fetch_memory(url, handle = handle) :
Failed to connect to srv2.stats-con.com port 443: Connection refused
con.opal·<-·opal.login(username·=·“administrator”,·password·=·“password”, url·=·“https://srv2.stats-con.com:8843”)
R console Error: curl::curl_fetch_memory(url, handle = handle) :
SSL certificate problem: self signed certificate
How do I login to an OPAL server running in Docker from an R analysis server (here on the same server, but not in Docker)?
Best, Wilmar
See recommended reverse proxy configuration (example is apache but it could also be nginx). If your institution does not provide a certificate, you can get one from Let’s encrypt.
Yannick
Hi,
I have installed and configured an nginx server with certbot to be able to use https now. I manually added the reverse proxy:
location / {
proxy_pass http://127.0.0.1:8843/;
}
However, R gives me this error: con.opal ← opal.login(username = “administrator”, password = “password”, url = “https://srv2.stats-con.com”)
Error in curl::curl_fetch_memory(url, handle = handle) :
> Received HTTP/0.9 when not allowed
However, curl gives me the same strange “Received HTTP/0.9 when not allowed” error.
root@fedsrv2:/var/www/html# curl --verbose -I https://srv2.stats-con.com
HEAD / HTTP/1.1
Host: srv2.stats-con.com
User-Agent: curl/7.74.0
Accept: /
TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
old SSL session ID is stale, removing
Received HTTP/0.9 when not allowed
Closing connection 0
TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (1) Received HTTP/0.9 when not allowed
The NGINX config file is updated as follows: default %>% certbot %>% added reverse proxy manually
How do I connect with curl and/or R/opalr to OPAL?
Best, Wilmar
nginx does not set the Host header by default (like Apache does). To avoid a CSRF check issue, you must specify it as follows:
location / {
proxy_pass http://127.0.0.1:8843/;
proxy_set_header Host $host;
}
Also, I think the proxied opal url is wrong as I suspect the port 8843 to be the one of the https entry point. Just use 8880 (or whatever you set in docker) to access via http.
Dear Yannick,
thanks! Seems to work now.
I have added to my NGINX default conf (see port mapping of docker image):
location / { proxy_pass http://127.0.0.1:8880/; proxy_set_header Host $host; }
/sudo:root@fedsrv2:/etc/nginx/sites-available/ #$ curl --verbose -I >https://srv2.stats-con.com
Trying 116.203.185.16:443…
Connected to srv2.stats-con.com (116.203.185.16) port 443 (#0)
ALPN, offering h2
ALPN, offering http/1.1
successfully set certificate verify locations:
CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
TLSv1.3 (OUT), TLS handshake, Client hello (1):
TLSv1.3 (IN), TLS handshake, Server hello (2):
TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
TLSv1.3 (IN), TLS handshake, Certificate (11):
TLSv1.3 (IN), TLS handshake, CERT verify (15):
TLSv1.3 (IN), TLS handshake, Finished (20):
TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
TLSv1.3 (OUT), TLS handshake, Finished (20):
SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
ALPN, server accepted to use http/1.1
Server certificate:
subject: CN=srv2.stats-con.com
start date: Aug 19 13:44:54 2022 GMT
expire date: Nov 17 13:44:53 2022 GMT
subjectAltName: host “srv2.stats-con.com” matched cert’s “srv2.stats-con.com”
issuer: C=US; O=Let’s Encrypt; CN=R3
SSL certificate verify ok.
HEAD / HTTP/1.1
Host: srv2.stats-con.com
User-Agent: curl/7.74.0
Accept: /TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
old SSL session ID is stale, removing
Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
HTTP/1.1 302 Found
< Server: nginx/1.18.0
Server: nginx/1.18.0
< Date: Sat, 20 Aug 2022 08:20:12 GMT
Date: Sat, 20 Aug 2022 08:20:12 GMT
< Content-Length: 0
Content-Length: 0
< Connection: keep-alive
Connection: keep-alive
< Location: http://srv2.stats-con.com/index.html
Location: http://srv2.stats-con.com/index.html
con.opal ← opal.login(username = “###”, password = “###”,
url = “https://srv2.stats-con.com”)
Thank you so much, Wilmar