Explore impact of Non Synonymous Filters

Summary of Findings:

This was an exploration of how TMB is affected by using nonsynonymous filters. It addresses part 1 of this OpenPBTA issue As far as the TMB comparisons go, it doesn’t matter if we use Variant_Classification filters or not. They are highly correlated no matter what and the same general TCGA to PBTA differences seem to persist.

Usage

If both run_caller_consensus_analysis-tcga.sh and run_caller_consensus_analysis-pbta.sh have been run, you can run this command. This will prepare the files needed for this notebook and run this notebook:

# bash run_explorations.sh

Table of Contents generated with DocToc

Read in the TMB files
- Read in PBTA data
- Read in TCGA data
Does the filter change a participant’s TMB?
- Plot PBTA data
- Plot TCGA data
Are some histologies affected more than others?
Does the filter affect the TCGA-PBTA comparison change?
Plot the ratio of the nonfilter/filter tmb
Plot the TMB plot with no filter data
- No filter TMB Plot
- With filter TMB Plot
Session Info

Setup

# Magrittr pipe
`%>%` <- dplyr::`%>%`

source(file.path("..", "..", "tmb-compare", "util", "cdf-plot-function.R"))

dir.create("plots", showWarnings = FALSE)

Read in the TMB files

Read in PBTA data

Import the tmb calculations that are not filtered by nonsynonymous classifications.

tmb_pbta_no_filter <- data.table::fread(file.path(
  "..",
  "results",
  "no_filter",
  "pbta-snv-mutation-tmb-coding.tsv"
)) %>%
  dplyr::filter(experimental_strategy != "Panel") %>%
  dplyr::mutate(filter = "no filter")

Repeat same steps for original tmb calculations (with filter).

tmb_pbta_with_filter <- data.table::fread(file.path(
  "..",
  "results",
  "consensus",
  "pbta-snv-mutation-tmb-coding.tsv"
)) %>%
  dplyr::filter(experimental_strategy != "Panel") %>%
  dplyr::mutate(filter = "filter")

Combine the unfilter and filtered data sets into one.

tmb_pbta <- dplyr::inner_join(tmb_pbta_no_filter,
  dplyr::select(tmb_pbta_with_filter, Tumor_Sample_Barcode, tmb),
  by = "Tumor_Sample_Barcode",
  suffix = c("_no_filter", "_filter")
)

Read in TCGA data

Repeat the same steps for the TCGA data.

tmb_tcga_no_filter <- data.table::fread(file.path(
  "..",
  "results",
  "no_filter",
  "tcga-snv-mutation-tmb-coding.tsv"
)) %>%
  dplyr::mutate(filter = "no filter")

tmb_tcga_with_filter <- data.table::fread(file.path(
  "..",
  "results",
  "consensus",
  "tcga-snv-mutation-tmb-coding.tsv"
)) %>%
  dplyr::mutate(filter = "filter")

Join these together.

tmb_tcga <- dplyr::inner_join(tmb_tcga_no_filter,
  dplyr::select(tmb_tcga_with_filter, Tumor_Sample_Barcode, tmb),
  by = "Tumor_Sample_Barcode",
  suffix = c("_no_filter", "_filter")
)

Does the filter change a participant’s TMB?

Plot PBTA data

Correlate first with Pearson’s.

cor.test(tmb_pbta$tmb_filter, tmb_pbta$tmb_no_filter)


    Pearson's product-moment correlation

data:  tmb_pbta$tmb_filter and tmb_pbta$tmb_no_filter
t = 2488.9, df = 934, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.9999143 0.9999337
sample estimates:
      cor 
0.9999246

pbta_cor_plot <- ggplot2::ggplot(
  tmb_pbta,
  ggplot2::aes(x = tmb_no_filter, y = tmb_filter, color = short_histology)
) +
  ggplot2::geom_point() +
  ggplot2::theme_classic() +
  ggplot2::theme(legend.position = "none")

pbta_cor_plot

This data is very skewed, let’s try a nonparametric correlation.

cor.test(tmb_pbta$tmb_filter, tmb_pbta$tmb_no_filter, method = "spearman")

Cannot compute exact p-value with ties


    Spearman's rank correlation rho

data:  tmb_pbta$tmb_filter and tmb_pbta$tmb_no_filter
S = 3031423, p-value < 2.2e-16
alternative hypothesis: true rho is not equal to 0
sample estimates:
      rho 
0.9778195

Still pretty good.

Plot TCGA data

cor.test(tmb_tcga$tmb_filter, tmb_tcga$tmb_no_filter)


    Pearson's product-moment correlation

data:  tmb_tcga$tmb_filter and tmb_tcga$tmb_no_filter
t = 4296.2, df = 316, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.9999893 0.9999931
sample estimates:
      cor 
0.9999914

tcga_cor_plot <- ggplot2::ggplot(
  tmb_tcga,
  ggplot2::aes(x = tmb_no_filter, y = tmb_filter, color = short_histology)
) +
  ggplot2::geom_point() +
  ggplot2::xlim(0, 5) +
  ggplot2::ylim(0, 5) +
  ggplot2::theme_classic() +
  ggplot2::theme(legend.position = "none")

tcga_cor_plot

Correlate with nonparametric correlation since we did that for PBTA.

cor.test(tmb_tcga$tmb_filter, tmb_tcga$tmb_no_filter, method = "spearman")

Cannot compute exact p-value with ties


    Spearman's rank correlation rho

data:  tmb_tcga$tmb_filter and tmb_tcga$tmb_no_filter
S = 60671, p-value < 2.2e-16
alternative hypothesis: true rho is not equal to 0
sample estimates:
      rho 
0.9886798

Are some histologies affected more than others?

Let’s make the same plots but by short_histology.

Plotting PBTA data, but setting the scales to “free” since the histologies have very different tmb ranges.

pbta_cor_plot + 
  ggplot2::facet_wrap(~short_histology, scales = "free")

Make the same plot for TCGA.

tcga_cor_plot + 
  ggplot2::facet_wrap(~short_histology)

Does the filter affect the TCGA-PBTA comparison change?

Put PBTA and TCGA data into one long format data frame.

all_data <- dplyr::bind_rows(list(tcga = tmb_tcga, pbta = tmb_pbta), .id = "dataset") %>%
  dplyr::select(tmb_filter, tmb_no_filter, dataset, Tumor_Sample_Barcode, short_histology) %>%
  tidyr::gather("filter", "tmb", -Tumor_Sample_Barcode, -dataset, -short_histology)

Plot the data overall.

ggplot2::ggplot(all_data, ggplot2::aes(x = dataset, y = tmb)) +
  ggforce::geom_sina() +
  ggplot2::theme_classic() +
  ggplot2::ylim(0, 10) +
  ggplot2::theme(legend.position = "none") +
  ggplot2::facet_wrap(~"filter")

Plot the ratio of the nonfilter/filter tmb

We’ll plot this by short_histology.

all_data %>% 
  tidyr::spread("filter","tmb") %>% 
  dplyr::mutate(ratio = tmb_filter/tmb_no_filter) %>% 
ggplot2::ggplot(ggplot2::aes(x = short_histology, y = ratio, color = short_histology)) + 
  ggplot2::geom_violin() + 
  ggplot2::theme_classic() + 
  ggplot2::theme(axis.text.x = ggplot2::element_text(angle = 45, hjust = 1), 
                 legend.position = "none") + 
  ggplot2::xlab("") + 
  ggplot2::ylab("tmb_filter/tmb_no_filter")

Plot the TMB plot with no filter data

This code is directly copied from tmb-compare-tcga/tmb-compare-tcga.Rmd.

pbta_plot <- cdf_plot(
  df = tmb_pbta_no_filter,
  plot_title = "PBTA",
  num_col = "tmb",
  group_col = "short_histology",
  color = "#3BC8A2",
  n_group = 5,
  x_lim = c(-1.2, 1.2),
  y_lim = c(0, 400),
  x_lab = "",
  y_lab = "Coding Mutations per Mb", 
  breaks = c(0, 3, 10, 30, 100, 300)
) +
  ggplot2::theme(
    strip.text.x = ggplot2::element_text(size = 12), 
    plot.margin = grid::unit(c(0.5, 0, 0.6, 0.5), "cm")
  )

the condition has length > 1 and only the first element will be used

tcga_plot <- cdf_plot(
  df = tmb_tcga_no_filter,
  plot_title = "TCGA (Adult)",
  num_col = "tmb",
  group_col = "short_histology",
  color = "#630882",
  n_group = 5,
  x_lim = c(-1.2, 1.2),
  y_lim = c(0, 400),
  x_lab = "",
  y_lab = "Coding Mutations per Mb",
  breaks = c()
) +
  ggplot2::theme(
    axis.title.y = ggplot2::element_blank(),
    axis.text.y = ggplot2::element_blank(),
    axis.ticks.y = ggplot2::element_blank(),
    strip.text.x = ggplot2::element_text(size = 9), 
    plot.margin = grid::unit(c(0.5, 1, 0.1, 0), "cm")
  )

the condition has length > 1 and only the first element will be used

# Put the plots together
tmb_plot <- cowplot::plot_grid(pbta_plot, tcga_plot,
  align = "v",
  axis = "left",
  rel_widths = c(2.5, 1),
  label_size = 12
)

Removed 3 rows containing missing values (geom_point).Removed 1 rows containing missing values (geom_point).

# Save the plot to a png
cowplot::save_plot(file.path("plots", "no_filter_tmb-cdf-pbta-tcga.png"),
  plot = tmb_plot, base_width = 35, base_height = 20, unit = "cm"
)

Print from png

No filter TMB Plot

With filter TMB Plot

## Session Info

sessionInfo()

R version 3.6.0 (2019-04-26)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS/LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C             
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1         pillar_1.4.2       compiler_3.6.0     RColorBrewer_1.1-2 plyr_1.8.4        
 [6] R.methodsS3_1.7.1  R.utils_2.9.0      base64enc_0.1-3    tools_3.6.0        digest_0.6.20     
[11] jsonlite_1.6       evaluate_0.14      tibble_2.1.3       gtable_0.3.0       pkgconfig_2.0.2   
[16] rlang_0.4.0        GGally_1.4.0       rstudioapi_0.10    yaml_2.2.0         xfun_0.8          
[21] colorblindr_0.1.0  dplyr_0.8.3        stringr_1.4.0      knitr_1.23         cowplot_0.9.4     
[26] grid_3.6.0         tidyselect_0.2.5   reshape_0.8.8      glue_1.3.1         data.table_1.12.2 
[31] R6_2.4.0           rmarkdown_1.13     polyclip_1.10-0    farver_1.1.0       tweenr_1.0.1      
[36] tidyr_0.8.3        ggplot2_3.2.0      purrr_0.3.2        reshape2_1.4.3     magrittr_1.5      
[41] MASS_7.3-51.4      scales_1.0.0       htmltools_0.3.6    assertthat_0.2.1   ggforce_0.2.2     
[46] colorspace_1.4-1   labeling_0.3       stringi_1.4.3      lazyeval_0.2.2     munsell_0.5.0     
[51] crayon_1.3.4       R.oo_1.22.0

LS0tCnRpdGxlOiAiRXhwbG9yZSBpbXBhY3Qgb2YgTm9uIFN5bm9ueW1vdXMgRmlsdGVycyIKb3V0cHV0OiAKICBodG1sX25vdGVib29rOgogICAgdG9jOiBUUlVFCiAgICB0b2NfZmxvYXQ6IFRSVUUKYXV0aG9yOiBDLiBTYXZvbmVuIGZvciBBTFNGIENDREwKZGF0ZTogMjAyMAotLS0KCiMjIyBTdW1tYXJ5IG9mIEZpbmRpbmdzOgoKVGhpcyB3YXMgYW4gZXhwbG9yYXRpb24gb2YgaG93IFRNQiBpcyBhZmZlY3RlZCBieSB1c2luZyBub25zeW5vbnltb3VzIGZpbHRlcnMuIApJdCBhZGRyZXNzZXMgcGFydCAxIG9mIFt0aGlzIE9wZW5QQlRBIGlzc3VlXShodHRwczovL2dpdGh1Yi5jb20vQWxleHNMZW1vbmFkZS9PcGVuUEJUQS1hbmFseXNpcy9pc3N1ZXMvNzI5KQpBcyBmYXIgYXMgdGhlIFRNQiBjb21wYXJpc29ucyBnbywgaXQgZG9lc24ndCBtYXR0ZXIgaWYgd2UgdXNlIGBWYXJpYW50X0NsYXNzaWZpY2F0aW9uYCBmaWx0ZXJzIG9yIG5vdC4gClRoZXkgYXJlIGhpZ2hseSBjb3JyZWxhdGVkIG5vIG1hdHRlciB3aGF0IGFuZCB0aGUgc2FtZSBnZW5lcmFsIFRDR0EgdG8gUEJUQSBkaWZmZXJlbmNlcyBzZWVtIHRvIHBlcnNpc3QuIAoKIyMjIFVzYWdlCgpJZiBib3RoIGBydW5fY2FsbGVyX2NvbnNlbnN1c19hbmFseXNpcy10Y2dhLnNoYCBhbmQgYHJ1bl9jYWxsZXJfY29uc2Vuc3VzX2FuYWx5c2lzLXBidGEuc2hgIGhhdmUgYmVlbiBydW4sIHlvdSBjYW4gcnVuIHRoaXMgY29tbWFuZC4gClRoaXMgd2lsbCBwcmVwYXJlIHRoZSBmaWxlcyBuZWVkZWQgZm9yIHRoaXMgbm90ZWJvb2sgYW5kIHJ1biB0aGlzIG5vdGVib29rOgoKYGBgCiMgYmFzaCBydW5fZXhwbG9yYXRpb25zLnNoCmBgYAoKPCEtLSBTVEFSVCBkb2N0b2MgZ2VuZXJhdGVkIFRPQyBwbGVhc2Uga2VlcCBjb21tZW50IGhlcmUgdG8gYWxsb3cgYXV0byB1cGRhdGUgLS0+CjwhLS0gRE9OJ1QgRURJVCBUSElTIFNFQ1RJT04sIElOU1RFQUQgUkUtUlVOIGRvY3RvYyBUTyBVUERBVEUgLS0+CioqVGFibGUgb2YgQ29udGVudHMqKiAgKmdlbmVyYXRlZCB3aXRoIFtEb2NUb2NdKGh0dHBzOi8vZ2l0aHViLmNvbS90aGxvcmVuei9kb2N0b2MpKgoKLSBbUmVhZCBpbiB0aGUgVE1CIGZpbGVzXSgjcmVhZC1pbi10aGUtdG1iLWZpbGVzKQogIC0gW1JlYWQgaW4gUEJUQSBkYXRhXSgjcmVhZC1pbi1wYnRhLWRhdGEpCiAgLSBbUmVhZCBpbiBUQ0dBIGRhdGFdKCNyZWFkLWluLXRjZ2EtZGF0YSkKLSBbRG9lcyB0aGUgZmlsdGVyIGNoYW5nZSBhIHBhcnRpY2lwYW50J3MgVE1CP10oI2RvZXMtdGhlLWZpbHRlci1jaGFuZ2UtYS1wYXJ0aWNpcGFudHMtdG1iKQogIC0gW1Bsb3QgUEJUQSBkYXRhXSgjcGxvdC1wYnRhLWRhdGEpCiAgLSBbUGxvdCBUQ0dBIGRhdGFdKCNwbG90LXRjZ2EtZGF0YSkKLSBbQXJlIHNvbWUgaGlzdG9sb2dpZXMgYWZmZWN0ZWQgbW9yZSB0aGFuIG90aGVycz9dKCNhcmUtc29tZS1oaXN0b2xvZ2llcy1hZmZlY3RlZC1tb3JlLXRoYW4tb3RoZXJzKQotIFtEb2VzIHRoZSBmaWx0ZXIgYWZmZWN0IHRoZSBUQ0dBLVBCVEEgY29tcGFyaXNvbiBjaGFuZ2U/XSgjZG9lcy10aGUtZmlsdGVyLWFmZmVjdC10aGUtdGNnYS1wYnRhLWNvbXBhcmlzb24tY2hhbmdlKQotIFtQbG90IHRoZSByYXRpbyBvZiB0aGUgbm9uZmlsdGVyL2ZpbHRlciB0bWJdKCNwbG90LXRoZS1yYXRpby1vZi10aGUtbm9uZmlsdGVyZmlsdGVyLXRtYikKLSBbUGxvdCB0aGUgVE1CIHBsb3Qgd2l0aCBubyBmaWx0ZXIgZGF0YV0oI3Bsb3QtdGhlLXRtYi1wbG90LXdpdGgtbm8tZmlsdGVyLWRhdGEpCiAgLSBbTm8gZmlsdGVyIFRNQiBQbG90XSgjbm8tZmlsdGVyLXRtYi1wbG90KQogIC0gW1dpdGggZmlsdGVyIFRNQiBQbG90XSgjd2l0aC1maWx0ZXItdG1iLXBsb3QpCi0gW1Nlc3Npb24gSW5mb10oI3Nlc3Npb24taW5mbykKCjwhLS0gRU5EIGRvY3RvYyBnZW5lcmF0ZWQgVE9DIHBsZWFzZSBrZWVwIGNvbW1lbnQgaGVyZSB0byBhbGxvdyBhdXRvIHVwZGF0ZSAtLT4KCiMjIFNldHVwCgpgYGB7cn0KIyBNYWdyaXR0ciBwaXBlCmAlPiVgIDwtIGRwbHlyOjpgJT4lYApgYGAKCmBgYHtyfQpzb3VyY2UoZmlsZS5wYXRoKCIuLiIsICIuLiIsICJ0bWItY29tcGFyZSIsICJ1dGlsIiwgImNkZi1wbG90LWZ1bmN0aW9uLlIiKSkKYGBgCgpgYGB7cn0KZGlyLmNyZWF0ZSgicGxvdHMiLCBzaG93V2FybmluZ3MgPSBGQUxTRSkKYGBgCgojIyBSZWFkIGluIHRoZSBUTUIgZmlsZXMKCiMjIyBSZWFkIGluIFBCVEEgZGF0YQoKSW1wb3J0IHRoZSB0bWIgY2FsY3VsYXRpb25zIHRoYXQgYXJlIG5vdCBmaWx0ZXJlZCBieSBub25zeW5vbnltb3VzIGNsYXNzaWZpY2F0aW9ucy4KCmBgYHtyfQp0bWJfcGJ0YV9ub19maWx0ZXIgPC0gZGF0YS50YWJsZTo6ZnJlYWQoZmlsZS5wYXRoKAogICIuLiIsCiAgInJlc3VsdHMiLAogICJub19maWx0ZXIiLAogICJwYnRhLXNudi1tdXRhdGlvbi10bWItY29kaW5nLnRzdiIKKSkgJT4lCiAgZHBseXI6OmZpbHRlcihleHBlcmltZW50YWxfc3RyYXRlZ3kgIT0gIlBhbmVsIikgJT4lCiAgZHBseXI6Om11dGF0ZShmaWx0ZXIgPSAibm8gZmlsdGVyIikKYGBgCgpSZXBlYXQgc2FtZSBzdGVwcyBmb3Igb3JpZ2luYWwgdG1iIGNhbGN1bGF0aW9ucyAod2l0aCBmaWx0ZXIpLgoKYGBge3J9CnRtYl9wYnRhX3dpdGhfZmlsdGVyIDwtIGRhdGEudGFibGU6OmZyZWFkKGZpbGUucGF0aCgKICAiLi4iLAogICJyZXN1bHRzIiwKICAiY29uc2Vuc3VzIiwKICAicGJ0YS1zbnYtbXV0YXRpb24tdG1iLWNvZGluZy50c3YiCikpICU+JQogIGRwbHlyOjpmaWx0ZXIoZXhwZXJpbWVudGFsX3N0cmF0ZWd5ICE9ICJQYW5lbCIpICU+JQogIGRwbHlyOjptdXRhdGUoZmlsdGVyID0gImZpbHRlciIpCmBgYAoKQ29tYmluZSB0aGUgdW5maWx0ZXIgYW5kIGZpbHRlcmVkIGRhdGEgc2V0cyBpbnRvIG9uZS4gCgpgYGB7cn0KdG1iX3BidGEgPC0gZHBseXI6OmlubmVyX2pvaW4odG1iX3BidGFfbm9fZmlsdGVyLAogIGRwbHlyOjpzZWxlY3QodG1iX3BidGFfd2l0aF9maWx0ZXIsIFR1bW9yX1NhbXBsZV9CYXJjb2RlLCB0bWIpLAogIGJ5ID0gIlR1bW9yX1NhbXBsZV9CYXJjb2RlIiwKICBzdWZmaXggPSBjKCJfbm9fZmlsdGVyIiwgIl9maWx0ZXIiKQopCmBgYAoKIyMjIFJlYWQgaW4gVENHQSBkYXRhCgpSZXBlYXQgdGhlIHNhbWUgc3RlcHMgZm9yIHRoZSBUQ0dBIGRhdGEuIAoKYGBge3J9CnRtYl90Y2dhX25vX2ZpbHRlciA8LSBkYXRhLnRhYmxlOjpmcmVhZChmaWxlLnBhdGgoCiAgIi4uIiwKICAicmVzdWx0cyIsCiAgIm5vX2ZpbHRlciIsCiAgInRjZ2Etc252LW11dGF0aW9uLXRtYi1jb2RpbmcudHN2IgopKSAlPiUKICBkcGx5cjo6bXV0YXRlKGZpbHRlciA9ICJubyBmaWx0ZXIiKQpgYGAKCmBgYHtyfQp0bWJfdGNnYV93aXRoX2ZpbHRlciA8LSBkYXRhLnRhYmxlOjpmcmVhZChmaWxlLnBhdGgoCiAgIi4uIiwKICAicmVzdWx0cyIsCiAgImNvbnNlbnN1cyIsCiAgInRjZ2Etc252LW11dGF0aW9uLXRtYi1jb2RpbmcudHN2IgopKSAlPiUKICBkcGx5cjo6bXV0YXRlKGZpbHRlciA9ICJmaWx0ZXIiKQpgYGAKCkpvaW4gdGhlc2UgdG9nZXRoZXIuIAoKYGBge3J9CnRtYl90Y2dhIDwtIGRwbHlyOjppbm5lcl9qb2luKHRtYl90Y2dhX25vX2ZpbHRlciwKICBkcGx5cjo6c2VsZWN0KHRtYl90Y2dhX3dpdGhfZmlsdGVyLCBUdW1vcl9TYW1wbGVfQmFyY29kZSwgdG1iKSwKICBieSA9ICJUdW1vcl9TYW1wbGVfQmFyY29kZSIsCiAgc3VmZml4ID0gYygiX25vX2ZpbHRlciIsICJfZmlsdGVyIikKKQpgYGAKCiMjIERvZXMgdGhlIGZpbHRlciBjaGFuZ2UgYSBwYXJ0aWNpcGFudCdzIFRNQj8KCiMjIyBQbG90IFBCVEEgZGF0YQoKQ29ycmVsYXRlIGZpcnN0IHdpdGggUGVhcnNvbidzLiAKCmBgYHtyfQpjb3IudGVzdCh0bWJfcGJ0YSR0bWJfZmlsdGVyLCB0bWJfcGJ0YSR0bWJfbm9fZmlsdGVyKQpgYGAKCmBgYHtyfQpwYnRhX2Nvcl9wbG90IDwtIGdncGxvdDI6OmdncGxvdCgKICB0bWJfcGJ0YSwKICBnZ3Bsb3QyOjphZXMoeCA9IHRtYl9ub19maWx0ZXIsIHkgPSB0bWJfZmlsdGVyLCBjb2xvciA9IHNob3J0X2hpc3RvbG9neSkKKSArCiAgZ2dwbG90Mjo6Z2VvbV9wb2ludCgpICsKICBnZ3Bsb3QyOjp0aGVtZV9jbGFzc2ljKCkgKwogIGdncGxvdDI6OnRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikKCnBidGFfY29yX3Bsb3QKYGBgCgpUaGlzIGRhdGEgaXMgdmVyeSBza2V3ZWQsIGxldCdzIHRyeSBhIG5vbnBhcmFtZXRyaWMgY29ycmVsYXRpb24uIAoKYGBge3J9CmNvci50ZXN0KHRtYl9wYnRhJHRtYl9maWx0ZXIsIHRtYl9wYnRhJHRtYl9ub19maWx0ZXIsIG1ldGhvZCA9ICJzcGVhcm1hbiIpCmBgYAoKU3RpbGwgcHJldHR5IGdvb2QuIAoKIyMjIFBsb3QgVENHQSBkYXRhCgpgYGB7cn0KY29yLnRlc3QodG1iX3RjZ2EkdG1iX2ZpbHRlciwgdG1iX3RjZ2EkdG1iX25vX2ZpbHRlcikKYGBgCgpgYGB7cn0KdGNnYV9jb3JfcGxvdCA8LSBnZ3Bsb3QyOjpnZ3Bsb3QoCiAgdG1iX3RjZ2EsCiAgZ2dwbG90Mjo6YWVzKHggPSB0bWJfbm9fZmlsdGVyLCB5ID0gdG1iX2ZpbHRlciwgY29sb3IgPSBzaG9ydF9oaXN0b2xvZ3kpCikgKwogIGdncGxvdDI6Omdlb21fcG9pbnQoKSArCiAgZ2dwbG90Mjo6eGxpbSgwLCA1KSArCiAgZ2dwbG90Mjo6eWxpbSgwLCA1KSArCiAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsKICBnZ3Bsb3QyOjp0aGVtZShsZWdlbmQucG9zaXRpb24gPSAibm9uZSIpCgp0Y2dhX2Nvcl9wbG90CmBgYAoKQ29ycmVsYXRlIHdpdGggbm9ucGFyYW1ldHJpYyBjb3JyZWxhdGlvbiBzaW5jZSB3ZSBkaWQgdGhhdCBmb3IgUEJUQS4gCgpgYGB7cn0KY29yLnRlc3QodG1iX3RjZ2EkdG1iX2ZpbHRlciwgdG1iX3RjZ2EkdG1iX25vX2ZpbHRlciwgbWV0aG9kID0gInNwZWFybWFuIikKYGBgCgojIyBBcmUgc29tZSBoaXN0b2xvZ2llcyBhZmZlY3RlZCBtb3JlIHRoYW4gb3RoZXJzPyAKCkxldCdzIG1ha2UgdGhlIHNhbWUgcGxvdHMgYnV0IGJ5ICBgc2hvcnRfaGlzdG9sb2d5YC4gCgpQbG90dGluZyBQQlRBIGRhdGEsIGJ1dCBzZXR0aW5nIHRoZSBzY2FsZXMgdG8gImZyZWUiIHNpbmNlIHRoZSBoaXN0b2xvZ2llcyBoYXZlIHZlcnkgZGlmZmVyZW50IHRtYiByYW5nZXMuIAoKYGBge3J9CnBidGFfY29yX3Bsb3QgKyAKICBnZ3Bsb3QyOjpmYWNldF93cmFwKH5zaG9ydF9oaXN0b2xvZ3ksIHNjYWxlcyA9ICJmcmVlIikKYGBgCgpNYWtlIHRoZSBzYW1lIHBsb3QgZm9yIFRDR0EuIAoKYGBge3J9CnRjZ2FfY29yX3Bsb3QgKyAKICBnZ3Bsb3QyOjpmYWNldF93cmFwKH5zaG9ydF9oaXN0b2xvZ3kpCmBgYAoKIyMgRG9lcyB0aGUgZmlsdGVyIGFmZmVjdCB0aGUgVENHQS1QQlRBIGNvbXBhcmlzb24gY2hhbmdlPwoKUHV0IFBCVEEgYW5kIFRDR0EgZGF0YSBpbnRvIG9uZSBsb25nIGZvcm1hdCBkYXRhIGZyYW1lLiAKCmBgYHtyfQphbGxfZGF0YSA8LSBkcGx5cjo6YmluZF9yb3dzKGxpc3QodGNnYSA9IHRtYl90Y2dhLCBwYnRhID0gdG1iX3BidGEpLCAuaWQgPSAiZGF0YXNldCIpICU+JQogIGRwbHlyOjpzZWxlY3QodG1iX2ZpbHRlciwgdG1iX25vX2ZpbHRlciwgZGF0YXNldCwgVHVtb3JfU2FtcGxlX0JhcmNvZGUsIHNob3J0X2hpc3RvbG9neSkgJT4lCiAgdGlkeXI6OmdhdGhlcigiZmlsdGVyIiwgInRtYiIsIC1UdW1vcl9TYW1wbGVfQmFyY29kZSwgLWRhdGFzZXQsIC1zaG9ydF9oaXN0b2xvZ3kpCmBgYAoKUGxvdCB0aGUgZGF0YSBvdmVyYWxsLiAKCmBgYHtyfQpnZ3Bsb3QyOjpnZ3Bsb3QoYWxsX2RhdGEsIGdncGxvdDI6OmFlcyh4ID0gZGF0YXNldCwgeSA9IHRtYikpICsKICBnZ2ZvcmNlOjpnZW9tX3NpbmEoKSArCiAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsKICBnZ3Bsb3QyOjp5bGltKDAsIDEwKSArCiAgZ2dwbG90Mjo6dGhlbWUobGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiKSArCiAgZ2dwbG90Mjo6ZmFjZXRfd3JhcCh+ImZpbHRlciIpIApgYGAKCiMjIFBsb3QgdGhlIHJhdGlvIG9mIHRoZSBub25maWx0ZXIvZmlsdGVyIHRtYgoKV2UnbGwgcGxvdCB0aGlzIGJ5IGBzaG9ydF9oaXN0b2xvZ3lgLiAKCmBgYHtyfQphbGxfZGF0YSAlPiUgCiAgdGlkeXI6OnNwcmVhZCgiZmlsdGVyIiwidG1iIikgJT4lIAogIGRwbHlyOjptdXRhdGUocmF0aW8gPSB0bWJfZmlsdGVyL3RtYl9ub19maWx0ZXIpICU+JSAKZ2dwbG90Mjo6Z2dwbG90KGdncGxvdDI6OmFlcyh4ID0gc2hvcnRfaGlzdG9sb2d5LCB5ID0gcmF0aW8sIGNvbG9yID0gc2hvcnRfaGlzdG9sb2d5KSkgKyAKICBnZ3Bsb3QyOjpnZW9tX3Zpb2xpbigpICsgCiAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsgCiAgZ2dwbG90Mjo6dGhlbWUoYXhpcy50ZXh0LnggPSBnZ3Bsb3QyOjplbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSwgCiAgICAgICAgICAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiKSArIAogIGdncGxvdDI6OnhsYWIoIiIpICsgCiAgZ2dwbG90Mjo6eWxhYigidG1iX2ZpbHRlci90bWJfbm9fZmlsdGVyIikKYGBgCgoKIyMgUGxvdCB0aGUgVE1CIHBsb3Qgd2l0aCBubyBmaWx0ZXIgZGF0YQoKVGhpcyBjb2RlIGlzIGRpcmVjdGx5IGNvcGllZCBmcm9tIGB0bWItY29tcGFyZS10Y2dhL3RtYi1jb21wYXJlLXRjZ2EuUm1kYC4KCmBgYHtyfQpwYnRhX3Bsb3QgPC0gY2RmX3Bsb3QoCiAgZGYgPSB0bWJfcGJ0YV9ub19maWx0ZXIsCiAgcGxvdF90aXRsZSA9ICJQQlRBIiwKICBudW1fY29sID0gInRtYiIsCiAgZ3JvdXBfY29sID0gInNob3J0X2hpc3RvbG9neSIsCiAgY29sb3IgPSAiIzNCQzhBMiIsCiAgbl9ncm91cCA9IDUsCiAgeF9saW0gPSBjKC0xLjIsIDEuMiksCiAgeV9saW0gPSBjKDAsIDQwMCksCiAgeF9sYWIgPSAiIiwKICB5X2xhYiA9ICJDb2RpbmcgTXV0YXRpb25zIHBlciBNYiIsIAogIGJyZWFrcyA9IGMoMCwgMywgMTAsIDMwLCAxMDAsIDMwMCkKKSArCiAgZ2dwbG90Mjo6dGhlbWUoCiAgICBzdHJpcC50ZXh0LnggPSBnZ3Bsb3QyOjplbGVtZW50X3RleHQoc2l6ZSA9IDEyKSwgCiAgICBwbG90Lm1hcmdpbiA9IGdyaWQ6OnVuaXQoYygwLjUsIDAsIDAuNiwgMC41KSwgImNtIikKICApCmBgYAoKYGBge3J9CnRjZ2FfcGxvdCA8LSBjZGZfcGxvdCgKICBkZiA9IHRtYl90Y2dhX25vX2ZpbHRlciwKICBwbG90X3RpdGxlID0gIlRDR0EgKEFkdWx0KSIsCiAgbnVtX2NvbCA9ICJ0bWIiLAogIGdyb3VwX2NvbCA9ICJzaG9ydF9oaXN0b2xvZ3kiLAogIGNvbG9yID0gIiM2MzA4ODIiLAogIG5fZ3JvdXAgPSA1LAogIHhfbGltID0gYygtMS4yLCAxLjIpLAogIHlfbGltID0gYygwLCA0MDApLAogIHhfbGFiID0gIiIsCiAgeV9sYWIgPSAiQ29kaW5nIE11dGF0aW9ucyBwZXIgTWIiLAogIGJyZWFrcyA9IGMoKQopICsKICBnZ3Bsb3QyOjp0aGVtZSgKICAgIGF4aXMudGl0bGUueSA9IGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwKICAgIGF4aXMudGV4dC55ID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLAogICAgYXhpcy50aWNrcy55ID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLAogICAgc3RyaXAudGV4dC54ID0gZ2dwbG90Mjo6ZWxlbWVudF90ZXh0KHNpemUgPSA5KSwgCiAgICBwbG90Lm1hcmdpbiA9IGdyaWQ6OnVuaXQoYygwLjUsIDEsIDAuMSwgMCksICJjbSIpCiAgKQpgYGAKCmBgYHtyfQojIFB1dCB0aGUgcGxvdHMgdG9nZXRoZXIKdG1iX3Bsb3QgPC0gY293cGxvdDo6cGxvdF9ncmlkKHBidGFfcGxvdCwgdGNnYV9wbG90LAogIGFsaWduID0gInYiLAogIGF4aXMgPSAibGVmdCIsCiAgcmVsX3dpZHRocyA9IGMoMi41LCAxKSwKICBsYWJlbF9zaXplID0gMTIKKQpgYGAKCmBgYHtyfQojIFNhdmUgdGhlIHBsb3QgdG8gYSBwbmcKY293cGxvdDo6c2F2ZV9wbG90KGZpbGUucGF0aCgicGxvdHMiLCAibm9fZmlsdGVyX3RtYi1jZGYtcGJ0YS10Y2dhLnBuZyIpLAogIHBsb3QgPSB0bWJfcGxvdCwgYmFzZV93aWR0aCA9IDM1LCBiYXNlX2hlaWdodCA9IDIwLCB1bml0ID0gImNtIgopCmBgYAoKUHJpbnQgZnJvbSBwbmcKCiMjIyBObyBmaWx0ZXIgVE1CIFBsb3QKIVtdKC4vcGxvdHMvbm9fZmlsdGVyX3RtYi1jZGYtcGJ0YS10Y2dhLnBuZykKCiMjIyBXaXRoIGZpbHRlciBUTUIgUGxvdAohW10oLi4vLi4vdG1iLWNvbXBhcmUvcGxvdHMvdG1iLWNkZi1wYnRhLXRjZ2EucG5nKQojIyBTZXNzaW9uIEluZm8KCmBgYHtyfQpzZXNzaW9uSW5mbygpCmBgYAo=