Last updated: 2019-02-24

workflowr checks: (Click a bullet for more information)
  • R Markdown file: up-to-date

    Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

  • Environment: empty

    Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

  • Seed: set.seed(20190115)

    The command set.seed(20190115) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

  • Session information: recorded

    Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

  • Repository version: bc45647

    Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

    Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:
    
    Ignored files:
        Ignored:    .DS_Store
        Ignored:    .Rhistory
        Ignored:    .Rproj.user/
        Ignored:    .sos/
        Ignored:    data/.DS_Store
        Ignored:    output/.DS_Store
    
    Untracked files:
        Untracked:  analysis/SuSiEzRProblem42.Rmd
        Untracked:  analysis/SuSiEzRProblem92.Rmd
        Untracked:  analysis/SusieZPerformanceRE8.Rmd
        Untracked:  docs/figure/test.Rmd/
        Untracked:  docs/small_data_42_sim_gaussian_36_get_sumstats_2_finemap_2_plot_finemap_2.plot_file.pdf
        Untracked:  docs/small_data_92_sim_gaussian_30_get_sumstats_2_finemap_2_plot_finemap_2.plot_file.pdf
        Untracked:  output/dscoutProblem475.rds
        Untracked:  output/dscoutProblem75.rds
        Untracked:  output/finemap_compare_random_data_null_dscout.rds
        Untracked:  output/finemap_compare_random_data_signal_dscout.rds
        Untracked:  output/finemap_compare_small_data_signal_dscout.rds
        Untracked:  output/finemap_compare_small_data_signal_dscout_RE8.rds
        Untracked:  output/random_data_100_sim_gaussian_null_1_get_sumstats_1_finemap_1.rds
        Untracked:  output/random_data_76.rds
        Untracked:  output/random_data_76_sim_gaussian_8.rds
        Untracked:  output/random_data_76_sim_gaussian_8_get_sumstats_1.rds
        Untracked:  output/small_data_42.rds
        Untracked:  output/small_data_42_sim_gaussian_36.rds
        Untracked:  output/small_data_42_sim_gaussian_36_get_sumstats_2.rds
        Untracked:  output/small_data_42_sim_gaussian_36_get_sumstats_2_susie_z_2.rds
        Untracked:  output/small_data_92.rds
        Untracked:  output/small_data_92_sim_gaussian_30.rds
        Untracked:  output/small_data_92_sim_gaussian_30_get_sumstats_2.rds
        Untracked:  output/small_data_92_sim_gaussian_30_get_sumstats_2_susie_z_2.rds
    
    Unstaged changes:
        Modified:   analysis/Problem475.Rmd
        Modified:   output/dsc_susie_z_v_output.rds
    
    
    Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
Expand here to see past versions:
    File Version Author Date Message
    Rmd bc45647 zouyuxin 2019-02-24 wflow_publish(“analysis/SusieZPerformanceRE3.Rmd”)


DAP-z is not stable, so we exclude it from the simulation. It has two problems:

  1. The algorithm doesn’t stop.

  2. It uses more than 6GB memories.

There is no credible set for signals in FINEMAP. We generate the credible set of causal signals as the union of the variables included in the smallest set of causal configurations that already covered 95% of the total posterior probability.

We use datasets from dsc-finemap, which is from GTExV8 genotypes. The genotype matrix X is 574 by 1001.

Simulation with signals

We simulate a gaussian y under various number of causal variables, total percentage of variance explained (PVE) and whether the signals have equal effect. The reason I control the effect size is that if we random generate the effect size for the signals, some signals have large effect size by chance. Therefore these signals have larger PVE.

We fit SuSiE with L = 5, FINEMAP with max 5 causals.

In SuSiE z, we truncate the eigenvalue of R at \(10^{-3}\), instead of \(10^{-8}\). There are some failed cases. Problem 92

library(dscrutils)
dscout = dscquery('output/finemap_compare_small_data_signal', target='method sim_gaussian.pve sim_gaussian.n_signal sim_gaussian.effect_weight score_susie.objective score_susie.converged score.total score.valid score.size score.signal_pip score_susie.purity score_susie.top score_susie.overlap ',group = c("score: score_susie score_finemap", "method: susie_z susie_z_init finemap"))

colnames(dscout) = c('DSC', 'method', 'output.file', 'pve', 'n_signal', 'effect_weight', 'score', 'objective', 'converged', 'total', 'valid', 'size', 'signal_pip', 'purity', 'top', 'overlap')
dscout$effect_weight[which(dscout$effect_weight == 'rep(1/n_signal, n_signal)')] = 'equal'
dscout$effect_weight[which(dscout$effect_weight != 'equal')] = 'notequal'
library(dplyr)

Attaching package: 'dplyr'
The following objects are masked from 'package:stats':

    filter, lag
The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union
library(knitr)
library(kableExtra)
library(susieR)
dscout = readRDS('output/finemap_compare_small_data_signal_dscout.rds')
dscout.susie = dscout[dscout$method == 'susie_z',]
dscout.susie.init = dscout[dscout$method == 'susie_z_init',]
dscout.finemap = dscout[dscout$method == 'finemap',]
  • Failed run:
success.susie = aggregate(converged~effect_weight+n_signal+pve, dscout.susie, sum)
times.susie = aggregate(DSC~effect_weight+n_signal+pve, dscout.susie, length)
success.times.susie = merge(times.susie, success.susie)
success.times.susie$Fail.susie = success.times.susie$DSC - success.times.susie$converged
colnames(success.times.susie)[colnames(success.times.susie) == 'DSC'] = 'times'
success.times.susie = success.times.susie[,-5]

success.susie.init = aggregate(converged~effect_weight+n_signal+pve, dscout.susie.init, sum)
times.susie.init = aggregate(DSC~effect_weight+n_signal+pve, dscout.susie.init, length)
success.times.susie.init = merge(times.susie, success.susie.init)
success.times.susie.init$Fail.susie.init = success.times.susie.init$DSC - success.times.susie.init$converged
colnames(success.times.susie.init)[colnames(success.times.susie.init) == 'DSC'] = 'times'
success.times.susie.init = success.times.susie.init[,-5]

success.times = Reduce(function(...) merge(...),
       list(success.times.susie, success.times.susie.init))

success.times %>% kable() %>% kable_styling(bootstrap_options = c("striped", "condensed", "responsive"), full_width = F) 
effect_weight n_signal pve times Fail.susie Fail.susie.init
equal 1 0.05 150 0 0
equal 1 0.10 150 0 0
equal 1 0.20 150 0 0
equal 1 0.60 150 0 0
equal 1 0.80 150 2 0
equal 10 0.05 150 0 0
equal 10 0.10 150 0 0
equal 10 0.20 150 0 0
equal 10 0.60 150 0 0
equal 10 0.80 150 0 0
equal 3 0.05 150 0 0
equal 3 0.10 150 0 0
equal 3 0.20 150 0 0
equal 3 0.60 150 0 0
equal 3 0.80 150 0 0
equal 5 0.05 150 0 0
equal 5 0.10 150 0 0
equal 5 0.20 150 0 0
equal 5 0.60 150 0 0
equal 5 0.80 150 0 0
notequal 1 0.05 150 0 0
notequal 1 0.10 150 0 0
notequal 1 0.20 150 0 0
notequal 1 0.60 150 0 0
notequal 1 0.80 150 2 0
notequal 10 0.05 150 0 0
notequal 10 0.10 150 0 0
notequal 10 0.20 150 0 0
notequal 10 0.60 150 0 0
notequal 10 0.80 150 2 2
notequal 3 0.05 150 0 0
notequal 3 0.10 150 0 0
notequal 3 0.20 150 0 0
notequal 3 0.60 150 1 1
notequal 3 0.80 150 2 1
notequal 5 0.05 150 0 0
notequal 5 0.10 150 0 0
notequal 5 0.20 150 0 0
notequal 5 0.60 150 1 1
notequal 5 0.80 150 3 3
  • Size of CS:
size.susie = aggregate(size~effect_weight+n_signal+pve, dscout.susie, FUN = function(x) round(mean(x), 2))
colnames(size.susie)[colnames(size.susie) == 'size'] <- 'size.susie'
size.susie.init = aggregate(size~effect_weight+n_signal+pve, dscout.susie.init, FUN = function(x) round(mean(x), 2))
colnames(size.susie.init)[colnames(size.susie.init) == 'size'] <- 'size.susie.init'
size.finemap = aggregate(size~effect_weight+n_signal+pve, dscout.finemap, FUN = function(x) round(mean(x), 2))
colnames(size.finemap)[colnames(size.finemap) == 'size'] <- 'size.finemap'

size = Reduce(function(...) merge(...),
       list(size.susie, size.susie.init, size.finemap))
size %>% kable() %>% kable_styling(bootstrap_options = c("striped", "condensed", "responsive"), full_width = F) 
effect_weight n_signal pve size.susie size.susie.init size.finemap
equal 1 0.05 7.84 9.11 998.07
equal 1 0.10 6.12 6.52 992.40
equal 1 0.20 11.11 10.75 968.32
equal 1 0.60 17.34 16.99 942.25
equal 1 0.80 4.60 4.02 563.46
equal 10 0.05 12.06 12.37 996.23
equal 10 0.10 11.34 11.02 942.31
equal 10 0.20 14.03 13.64 681.95
equal 10 0.60 25.43 25.44 229.12
equal 10 0.80 18.72 18.59 147.27
equal 3 0.05 9.97 10.53 990.67
equal 3 0.10 7.77 9.41 905.46
equal 3 0.20 12.04 12.30 702.05
equal 3 0.60 20.63 20.94 427.31
equal 3 0.80 12.26 12.18 174.79
equal 5 0.05 12.20 12.19 992.99
equal 5 0.10 12.87 12.82 942.01
equal 5 0.20 11.99 12.01 580.85
equal 5 0.60 19.21 19.22 165.55
equal 5 0.80 18.91 18.78 99.75
notequal 1 0.05 7.84 9.11 998.08
notequal 1 0.10 6.12 6.52 992.31
notequal 1 0.20 11.11 10.75 967.49
notequal 1 0.60 17.34 16.99 941.80
notequal 1 0.80 4.60 4.02 565.49
notequal 10 0.05 9.04 8.87 992.35
notequal 10 0.10 9.44 8.79 967.02
notequal 10 0.20 14.33 14.41 902.51
notequal 10 0.60 19.50 19.50 746.36
notequal 10 0.80 12.47 12.25 664.63
notequal 3 0.05 12.83 14.16 993.11
notequal 3 0.10 11.06 12.86 972.49
notequal 3 0.20 15.63 15.77 885.79
notequal 3 0.60 22.00 22.05 712.27
notequal 3 0.80 12.33 12.07 548.23
notequal 5 0.05 10.44 10.91 994.36
notequal 5 0.10 9.64 9.62 979.18
notequal 5 0.20 12.39 12.28 927.49
notequal 5 0.60 19.27 19.27 855.17
notequal 5 0.80 13.61 13.61 644.88
  • Purity of CS:
purity.susie = aggregate(purity~effect_weight+n_signal+pve, dscout.susie, FUN = function(x) round(mean(x), 3))
colnames(purity.susie)[colnames(purity.susie) == 'purity'] <- 'purity.susie'
purity.susie.init = aggregate(purity~effect_weight+n_signal+pve, dscout.susie.init, FUN = function(x) round(mean(x), 3))
colnames(purity.susie.init)[colnames(purity.susie.init) == 'purity'] <- 'purity.susie.init'

purity = Reduce(function(...) merge(...),
       list(purity.susie, purity.susie.init))
purity %>% kable() %>% kable_styling(bootstrap_options = c("striped", "condensed", "responsive"), full_width = F) 
effect_weight n_signal pve purity.susie purity.susie.init
equal 1 0.05 0.927 0.881
equal 1 0.10 0.968 0.970
equal 1 0.20 0.978 0.987
equal 1 0.60 0.906 0.914
equal 1 0.80 0.666 0.683
equal 10 0.05 0.769 0.760
equal 10 0.10 0.911 0.893
equal 10 0.20 0.947 0.952
equal 10 0.60 0.943 0.944
equal 10 0.80 0.893 0.892
equal 3 0.05 0.836 0.823
equal 3 0.10 0.939 0.931
equal 3 0.20 0.965 0.963
equal 3 0.60 0.947 0.946
equal 3 0.80 0.863 0.866
equal 5 0.05 0.787 0.786
equal 5 0.10 0.917 0.916
equal 5 0.20 0.959 0.959
equal 5 0.60 0.959 0.960
equal 5 0.80 0.905 0.904
notequal 1 0.05 0.927 0.881
notequal 1 0.10 0.968 0.970
notequal 1 0.20 0.978 0.987
notequal 1 0.60 0.906 0.914
notequal 1 0.80 0.666 0.683
notequal 10 0.05 0.937 0.939
notequal 10 0.10 0.949 0.960
notequal 10 0.20 0.953 0.956
notequal 10 0.60 0.955 0.956
notequal 10 0.80 0.870 0.871
notequal 3 0.05 0.918 0.917
notequal 3 0.10 0.941 0.943
notequal 3 0.20 0.955 0.951
notequal 3 0.60 0.941 0.940
notequal 3 0.80 0.828 0.832
notequal 5 0.05 0.913 0.913
notequal 5 0.10 0.951 0.951
notequal 5 0.20 0.945 0.946
notequal 5 0.60 0.944 0.944
notequal 5 0.80 0.833 0.833
  • Power:
valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.susie, sum)
total = aggregate(DSC~ effect_weight + n_signal + pve, dscout.susie, length)
total$total_true = total$DSC * total$n_signal
power.susie = merge(valid, total)
power.susie$power.susie = round(power.susie$valid/(power.susie$total_true), 3)
colnames(power.susie)[colnames(power.susie) == 'valid'] <- 'valid.susie'

valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.susie.init, sum)
total = aggregate(DSC~ effect_weight + n_signal + pve, dscout.susie.init, length)
total$total_true = total$DSC * total$n_signal
power.susie.init = merge(valid, total)
power.susie.init$power.susie.init = round(power.susie.init$valid/(power.susie.init$total_true), 3)
colnames(power.susie.init)[colnames(power.susie.init) == 'valid'] <- 'valid.susie.init'

valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.finemap, sum)
total = aggregate(DSC ~ effect_weight + n_signal + pve, dscout.finemap, length)
total$total_true = total$DSC * total$n_signal
power.finemap = merge(valid, total)
power.finemap$power.finemap = round(power.finemap$valid/(power.finemap$total_true),3)
colnames(power.finemap)[colnames(power.finemap) == 'valid'] <- 'valid.finemap'

power = Reduce(function(...) merge(...),
       list(power.susie, power.susie.init, power.finemap))
power = power[,-4]
power %>% kable() %>% kable_styling(bootstrap_options = c("striped", "condensed", "responsive","bordered"), full_width = F) %>% add_header_above(c(" ", " ", " "," ", "SuSiE z" = 2, "SuSiE z init" = 2,"FINEMAP" = 2)) %>% column_spec(c(6, 8, 10), bold = T)
SuSiE z
SuSiE z init
FINEMAP
effect_weight n_signal pve total_true valid.susie power.susie valid.susie.init power.susie.init valid.finemap power.finemap
equal 1 0.05 150 110 0.733 110 0.733 150 1.000
equal 1 0.10 150 120 0.800 123 0.820 150 1.000
equal 1 0.20 150 141 0.940 141 0.940 150 1.000
equal 1 0.60 150 141 0.940 141 0.940 139 0.927
equal 1 0.80 150 101 0.673 104 0.693 140 0.933
equal 10 0.05 1500 100 0.067 105 0.070 1494 0.996
equal 10 0.10 1500 198 0.132 193 0.129 1455 0.970
equal 10 0.20 1500 327 0.218 396 0.264 1320 0.880
equal 10 0.60 1500 387 0.258 655 0.437 1104 0.736
equal 10 0.80 1500 350 0.233 573 0.382 1004 0.669
equal 3 0.05 450 125 0.278 111 0.247 449 0.998
equal 3 0.10 450 237 0.527 230 0.511 446 0.991
equal 3 0.20 450 307 0.682 311 0.691 446 0.991
equal 3 0.60 450 329 0.731 336 0.747 438 0.973
equal 3 0.80 450 304 0.676 311 0.691 412 0.916
equal 5 0.05 750 119 0.159 118 0.157 748 0.997
equal 5 0.10 750 277 0.369 275 0.367 743 0.991
equal 5 0.20 750 432 0.576 435 0.580 715 0.953
equal 5 0.60 750 512 0.683 513 0.684 686 0.915
equal 5 0.80 750 473 0.631 475 0.633 653 0.871
notequal 1 0.05 150 110 0.733 110 0.733 150 1.000
notequal 1 0.10 150 120 0.800 123 0.820 150 1.000
notequal 1 0.20 150 141 0.940 141 0.940 150 1.000
notequal 1 0.60 150 141 0.940 141 0.940 138 0.920
notequal 1 0.80 150 101 0.673 104 0.693 142 0.947
notequal 10 0.05 1500 137 0.091 139 0.093 1492 0.995
notequal 10 0.10 1500 163 0.109 159 0.106 1471 0.981
notequal 10 0.20 1500 203 0.135 211 0.141 1414 0.943
notequal 10 0.60 1500 194 0.129 199 0.133 1323 0.882
notequal 10 0.80 1500 156 0.104 154 0.103 1246 0.831
notequal 3 0.05 450 122 0.271 122 0.271 449 0.998
notequal 3 0.10 450 154 0.342 142 0.316 449 0.998
notequal 3 0.20 450 193 0.429 186 0.413 443 0.984
notequal 3 0.60 450 177 0.393 175 0.389 428 0.951
notequal 3 0.80 450 135 0.300 132 0.293 400 0.889
notequal 5 0.05 750 122 0.163 122 0.163 748 0.997
notequal 5 0.10 750 151 0.201 150 0.200 743 0.991
notequal 5 0.20 750 200 0.267 198 0.264 732 0.976
notequal 5 0.60 750 182 0.243 182 0.243 716 0.955
notequal 5 0.80 750 145 0.193 145 0.193 667 0.889
  • FDR
valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.susie, sum)
total = aggregate(total~ effect_weight + n_signal + pve, dscout.susie, sum)
fdr.susie = merge(valid, total)
fdr.susie$fdr.susie = round((fdr.susie$total - fdr.susie$valid)/fdr.susie$total, 3)
colnames(fdr.susie)[colnames(fdr.susie) == 'valid'] <- 'valid.susie'
fdr.susie = fdr.susie[,-5]

valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.susie.init, sum)
total = aggregate(total~ effect_weight + n_signal + pve, dscout.susie.init, sum)
fdr.susie.init = merge(valid, total)
fdr.susie.init$fdr.susie.init = round((fdr.susie.init$total - fdr.susie.init$valid)/fdr.susie.init$total, 3)
colnames(fdr.susie.init)[colnames(fdr.susie.init) == 'valid'] <- 'valid.susie.init'
fdr.susie.init = fdr.susie.init[,-5]

valid = aggregate(valid ~ effect_weight + n_signal + pve, dscout.finemap, sum)
total = aggregate(size ~ effect_weight + n_signal + pve, dscout.finemap, sum)
fdr.finemap = merge(valid, total)
fdr.finemap$fdr.finemap = round((fdr.finemap$size - fdr.finemap$valid)/fdr.finemap$size, 3)
colnames(fdr.finemap)[colnames(fdr.finemap) == 'valid'] <- 'valid.finemap'
fdr.finemap = fdr.finemap[,-5]

fdr = Reduce(function(...) merge(...),
       list(fdr.susie, fdr.susie.init, fdr.finemap))

fdr %>% kable() %>% kable_styling(bootstrap_options = c("striped", "condensed", "responsive","bordered"), full_width = F) %>% add_header_above(c(" ", " ", " ", "SuSiE z" = 2, "SuSiE z init" = 2,"FINEMAP" = 2))
SuSiE z
SuSiE z init
FINEMAP
effect_weight n_signal pve valid.susie fdr.susie valid.susie.init fdr.susie.init valid.finemap fdr.finemap
equal 1 0.05 110 0.534 110 0.231 150 0.999
equal 1 0.10 120 0.444 123 0.180 150 0.999
equal 1 0.20 141 0.217 141 0.060 150 0.999
equal 1 0.60 141 0.212 141 0.000 139 0.999
equal 1 0.80 101 0.409 104 0.000 140 0.998
equal 10 0.05 100 0.628 105 0.591 1494 0.990
equal 10 0.10 198 0.519 193 0.545 1455 0.990
equal 10 0.20 327 0.402 396 0.437 1320 0.987
equal 10 0.60 387 0.273 655 0.234 1104 0.968
equal 10 0.80 350 0.236 573 0.201 1004 0.955
equal 3 0.05 125 0.604 111 0.554 449 0.997
equal 3 0.10 237 0.441 230 0.361 446 0.997
equal 3 0.20 307 0.315 311 0.228 446 0.996
equal 3 0.60 329 0.167 336 0.123 438 0.993
equal 3 0.80 304 0.172 311 0.114 412 0.984
equal 5 0.05 119 0.584 118 0.586 748 0.995
equal 5 0.10 277 0.409 275 0.415 743 0.995
equal 5 0.20 432 0.281 435 0.280 715 0.992
equal 5 0.60 512 0.142 513 0.142 686 0.972
equal 5 0.80 473 0.131 475 0.128 653 0.956
notequal 1 0.05 110 0.534 110 0.231 150 0.999
notequal 1 0.10 120 0.444 123 0.180 150 0.999
notequal 1 0.20 141 0.217 141 0.060 150 0.999
notequal 1 0.60 141 0.212 141 0.000 138 0.999
notequal 1 0.80 101 0.409 104 0.000 142 0.998
notequal 10 0.05 137 0.477 139 0.469 1492 0.990
notequal 10 0.10 163 0.455 159 0.463 1471 0.990
notequal 10 0.20 203 0.316 211 0.310 1414 0.990
notequal 10 0.60 194 0.174 199 0.191 1323 0.988
notequal 10 0.80 156 0.235 154 0.312 1246 0.988
notequal 3 0.05 122 0.540 122 0.463 449 0.997
notequal 3 0.10 154 0.462 142 0.396 449 0.997
notequal 3 0.20 193 0.306 186 0.271 443 0.997
notequal 3 0.60 177 0.088 175 0.054 428 0.996
notequal 3 0.80 135 0.220 132 0.143 400 0.995
notequal 5 0.05 122 0.472 122 0.472 748 0.995
notequal 5 0.10 151 0.415 150 0.414 743 0.995
notequal 5 0.20 200 0.270 198 0.272 732 0.995
notequal 5 0.60 182 0.180 182 0.180 716 0.994
notequal 5 0.80 145 0.341 145 0.344 667 0.993

Session information

sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS  10.14.3

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] susieR_0.6.4.0438 kableExtra_1.0.1  knitr_1.20        dplyr_0.7.8      

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0        highr_0.7         compiler_3.5.1   
 [4] pillar_1.3.1      git2r_0.24.0      workflowr_1.1.1  
 [7] bindr_0.1.1       R.methodsS3_1.7.1 R.utils_2.7.0    
[10] tools_3.5.1       digest_0.6.18     lattice_0.20-38  
[13] evaluate_0.12     tibble_2.0.1      viridisLite_0.3.0
[16] pkgconfig_2.0.2   rlang_0.3.1       Matrix_1.2-15    
[19] rstudioapi_0.9.0  yaml_2.2.0        bindrcpp_0.2.2   
[22] stringr_1.3.1     httr_1.4.0        xml2_1.2.0       
[25] hms_0.4.2         grid_3.5.1        webshot_0.5.1    
[28] rprojroot_1.3-2   tidyselect_0.2.5  glue_1.3.0       
[31] R6_2.3.0          rmarkdown_1.11    purrr_0.2.5      
[34] readr_1.3.1       magrittr_1.5      whisker_0.3-2    
[37] backports_1.1.3   scales_1.0.0      htmltools_0.3.6  
[40] assertthat_0.2.0  rvest_0.3.2       colorspace_1.4-0 
[43] stringi_1.2.4     munsell_0.5.0     crayon_1.3.4     
[46] R.oo_1.22.0      

This reproducible R Markdown analysis was created with workflowr 1.1.1