我的第一R包- 生信配置文件大全

大家好，我是李剑峰，生信技能树论坛的VIP小编，目前在上海交通大学医学院附属瑞金医院进行研究生阶段的学习，主要研究方向是生物信息学、医学信息学、大数据综合分析、临床诊断数据分析、DNA-seq， RNA-seq数据分析。很高兴又和读者朋友见面啦，有心的同学应该还记得我的上一篇教程：

Pecan Data Portal 系列教程（一）

很不幸，该网站被我们伟大的长城给屏蔽了，所以系列教程暂时夭折，不过，分享的脚步不会停下，下面介绍我的一个R包！

简介

configr 是我上传到CRAN的第一个R包，主要功能是解析和生成配置文件（json/ini/yaml/toml），分别用到了jsonlite，ini， yaml和RcppTOML。

configr是一个整合的并且进行了解析扩展的R包（开发动机主要是我比较烦记函数，另外解析配置文件之后很多参数还要做进一步处理），本教程将介绍常见的几种配置文件格式，以及configr的基本用法。

常见的配置文件类型

在生物信息学工具或者方法开发过程中，给用户提供一个简洁明了的配置文件进行自定义配置是一个非常好的选择。所以，为了更好的编写和解析配置文件，大家首先就要熟悉目前R语言中常用的配置文件解析工具，我下面列出了目前主要的几种配置文件格式供大家参考：

json

  1. {

  2.  "default": {

  3.    "debug": "{{debug}} {{debug2}}"

  4.  },

  5.  "comments": {

  6.    "version": "0.2.3"

  7.  }

  8. }

参考：json.org, json examples, json-wiki

ini

  1. [default]

  2. debug = {{debug}} {{debug2}}

  3. [comments]

  4. version = 0.2.3

参考：ini-wiki

yaml

  1. default:

  2.  debug: '{{debug}} {{debug2}}'

  3. comments:

  4.  version: 0.2.3

参考：yaml.org, yaml-wiki

toml

  1. # This is a TOML document. Jianfeng.

  2. title = "TOML Example"

  3. [default]

  4. debug = "{{debug}} {{debug2}}"

  5. [comments]

  6. version = "0.2.3"

  7. [comments.ljf]

  8. content = "Hello World!"

参考：toml-github, toml-wiki

xml

  1. <?xml version="1.0" encoding="ISO-8859-1"?>

  2. <!--  Copyright w3school.com.cn -->

  3. <note>

  4.    <to>George</to>

  5.    <from>John</from>

  6.    <heading>Reminder</heading>

  7.    <body>Don't forget the meeting!</body>

  8. </note>

参考：XML-wiki

用法

configr选择支持了json, ini, yaml, toml四种配置文件格式，下面将主要讲一下configr的基本用法，主要分为一下几块内容：配置文件格式识别、配置文件读取、配置文件的格式转换、配置文件扩展解析

格式识别

  1. # 获取R包configr中内置的四种配置文件

  2. library(configr)

  3. config.json <- system.file("extdata", "config.json", package = "configr")

  4. config.ini <- system.file("extdata", "config.ini", package = "configr")

  5. config.yaml <- system.file("extdata", "config.yaml", package = "configr")

  6. config.toml <- system.file("extdata", "config.toml", package = "configr")

  7. # 配置文件格式识别相关函数

  8. is.json.file(config.json)

  1. ## [1] TRUE

  1. is.toml.file(config.toml)

  1. ## [1] TRUE

  1. is.ini.file(config.ini)

  1. ## [1] TRUE

  1. is.yaml.file(config.yaml)

  1. ## [1] TRUE

  1. get.config.type(config.json)

  1. ## [1] "json"

  1. get.config.type(config.yaml)

  1. ## [1] "yaml"

  1. get.config.type(config.ini)

  1. ## [1] "ini"

  1. get.config.type(config.toml)

  1. ## [1] "toml"

配置文件读取

  1. # 获取配置文件中主键名称

  2. eval.config.sections(config.ini)

  1. ## [1] "default"            "comments"          

  2. ## [3] "extra_list_parse"   "other_config_parse"

  3. ## [5] "rcmd_parse"         "bash_parse"        

  4. ## [7] "mulitple_parse"     "glue_parse"

  1. eval.config.sections(config.toml)

  1. ## [1] "bash_parse"         "comments"          

  2. ## [3] "default"            "extra_list_parse"  

  3. ## [5] "glue_parse"         "mulitple_parse"    

  4. ## [7] "other_config_parse" "title"

  1. # 读取配置文件生成R中的列表

  2. # 注意：可以直接传递fromJSON/read.ini/readLines/yaml.load相关参数给read.config

  3. read.config(file = config.toml)

  1. ## List of 8

  2. ##  $ bash_parse        :List of 2

  3. ##   ..$ parsed: chr "bash"

  4. ##   ..$ raw   : chr "#>#echo bash#<#"

  5. ##  $ comments          :List of 1

  6. ##   ..$ version: chr "0.2.3"

  7. ##  $ default           :List of 1

  8. ##   ..$ debug: chr "{{debug}} {{debug2}}"

  9. ##  $ extra_list_parse  :List of 2

  10. ##   ..$ parsed: chr "1"

  11. ##   ..$ raw   : chr "{{yes}}"

  12. ##  $ glue_parse        :List of 4

  13. ##   ..$ parsed_1: chr [1:10] "1" "2" "3" "4" ...

  14. ##   ..$ parsed_2: int [1:10] 1 2 3 4 5 6 7 8 9 10

  15. ##   ..$ raw_1   : chr "!!glue {1:10}"

  16. ##   ..$ raw_2   : chr "!!glue_numeric {1:10}"

  17. ##  $ mulitple_parse    :List of 2

  18. ##   ..$ parsed: chr "configr, configr, yes, 1, config, config, no, 0"

  19. ##   ..$ raw   : chr "@>@str_replace('config','g$','gr')@<@, #>#echo configr#<#, {{key:yes_flag}}, {{yes}}, @>@str_replace('configr',"| __truncated__

  20. ##  $ other_config_parse:List of 2

  21. ##   ..$ parsed: chr "yes no"

  22. ##   ..$ raw   : chr "{{key:yes_flag}} {{key:no_flag}}"

  23. ##  $ title             : chr "TOML Example"

  1. # Get the same obj with config package, only get the 'default

  2. # or R_CONFIG_ACTIVE config sets' in config.cfg or

  3. # R_CONFIGFILE_ACTIVE

  4. eval.config(file = config.yaml)

  1. ## $debug

  2. ## [1] "{{debug}} {{debug2}}"

  3. ##

  4. ## attr(,"config")

  5. ## [1] "default"

  6. ## attr(,"configtype")

  7. ## [1] "yaml"

  8. ## attr(,"file")

  9. ## [1] "/home/ljf/Rlibrary/configr/extdata/config.yaml"

  1. # Read designated section

  2. eval.config(file = config.json, config = "comments")

  1. ## $version

  2. ## [1] "0.2.3"

  3. ##

  4. ## attr(,"config")

  5. ## [1] "comments"

  6. ## attr(,"configtype")

  7. ## [1] "json"

  8. ## attr(,"file")

  9. ## [1] "/home/ljf/Rlibrary/configr/extdata/config.json"

  1. # Read designated section with its one value

  2. eval.config(file = config.ini, config = "comments", value = "version")

  1. ## [1] "0.2.3"

  1. # eval.config.merge 可以合并几个主键并减少配置文件的层数

  2. eval.config.merge(file = config.json, sections = c("default",

  3.  "comments"))

  1. ## $debug

  2. ## [1] "{{debug}} {{debug2}}"

  3. ##

  4. ## $version

  5. ## [1] "0.2.3"

  6. ##

  7. ## attr(,"config")

  8. ## [1] "default"  "comments"

  9. ## attr(,"configtype")

  10. ## [1] "json"

  11. ## attr(,"file")

  12. ## [1] "/home/ljf/Rlibrary/configr/extdata/config.json"

  1. eval.config.merge(file = config.toml, sections = c("default",

  2.  "comments"))

  1. ## $debug

  2. ## [1] "{{debug}} {{debug2}}"

  3. ##

  4. ## $version

  5. ## [1] "0.2.3"

  6. ##

  7. ## attr(,"config")

  8. ## [1] "default"  "comments"

  9. ## attr(,"configtype")

  10. ## [1] "toml"

  11. ## attr(,"file")

  12. ## [1] "/home/ljf/Rlibrary/configr/extdata/config.toml"

  1. # fetch.config可以导入http:// ftp://以及本地文件，

  2. # 它会将这些文件进行收集和读取，然后生成一个合并的R列表对象

  3. links <- c("https://raw.githubusercontent.com/JhuangLab/BioInstaller/master/inst/extdata/config/db/db_annovar.toml",

  4.  "https://raw.githubusercontent.com/JhuangLab/BioInstaller/master/inst/extdata/config/db/db_main.toml",

  5.  system.file("extdata", "config.toml", package = "configr"))

  6. x <- fetch.config(links)

  7. x[c(1:5, length(x))]

  1. ## $db_annovar_1000g

  2. ## $db_annovar_1000g$buildver_available

  3. ## $db_annovar_1000g$buildver_available$`1000g`

  4. ## [1] "hg18"

  5. ##

  6. ## $db_annovar_1000g$buildver_available$`1000g2010`

  7. ## [1] "hg18"

  8. ##

  9. ## $db_annovar_1000g$buildver_available$`1000g2012apr`

  10. ## [1] "hg19" "hg18"

  11. ##

  12. ## $db_annovar_1000g$buildver_available$`1000g2012jul`

  13. ## [1] "hg18"

  14. ##

  15. ## $db_annovar_1000g$buildver_available$`1000g2014oct`

  16. ## [1] "hg38" "hg19" "hg18"

  17. ##

  18. ## $db_annovar_1000g$buildver_available$`1000g2015aug`

  19. ## [1] "hg38" "hg19"

  20. ##

  21. ## $db_annovar_1000g$buildver_available$other

  22. ## [1] "hg19"

  23. ##

  24. ##

  25. ## $db_annovar_1000g$description

  26. ## [1] "alternative allele frequency data in 1000 Genomes Project"

  27. ##

  28. ## $db_annovar_1000g$source_url

  29. ## [1] "http://www.openbioinformatics.org/annovar/download/{{buildver}}_{{version}}.zip"

  30. ##

  31. ## $db_annovar_1000g$version_available

  32. ##  [1] "1000g2015aug" "1000g2014oct" "1000g2014sep"

  33. ##  [4] "1000g2014aug" "1000g2012apr" "1000g2012feb"

  34. ##  [7] "1000g2011may" "1000g2010nov" "1000g2012apr"

  35. ## [10] "1000g2010jul" "1000g2010"    "1000g"      

  36. ##

  37. ## $db_annovar_1000g$version_newest

  38. ## [1] "1000g2015aug"

  39. ##

  40. ##

  41. ## $db_annovar_1000g_sqlite

  42. ## $db_annovar_1000g_sqlite$buildver_available

  43. ## [1] "hg19"

  44. ##

  45. ## $db_annovar_1000g_sqlite$install

  46. ## [1] "#R#for(i in c('all', 'afr', 'eas', 'eur', 'sas', 'amr')) {\\n  x <- set.1000g.db(sprintf('{{version}}_%s', i), '{{buildver}}', \\\"sql\\\");\\n  params <- list(sql.file = x, sqlite.path = str_replace(x, '.sql$', ''));\\n  do.call(sql2sqlite, params)\\n}\\n#R#"

  47. ##

  48. ## $db_annovar_1000g_sqlite$source_url

  49. ## [1] "http://bioinfo.rjh.com.cn/download/annovarR/humandb/{{buildver}}_{{version}}.tar.gz"

  50. ##

  51. ## $db_annovar_1000g_sqlite$version_available

  52. ## [1] "1000g2015aug"

  53. ##

  54. ## $db_annovar_1000g_sqlite$version_newest

  55. ## [1] "1000g2015aug"

  56. ##

  57. ##

  58. ## $db_annovar_avsift

  59. ## $db_annovar_avsift$buildver_available

  60. ## [1] "hg19" "hg18"

  61. ##

  62. ## $db_annovar_avsift$decompress

  63. ## [1] TRUE TRUE

  64. ##

  65. ## $db_annovar_avsift$description

  66. ## [1] "whole-exome SIFT scores for non-synonymous variants (obselete and should not be uesd any more)"

  67. ##

  68. ## $db_annovar_avsift$source_url

  69. ## [1] "http://www.openbioinformatics.org/annovar/download/{{buildver}}_{{version}}.txt.gz"    

  70. ## [2] "http://www.openbioinformatics.org/annovar/download/{{buildver}}_{{version}}.txt.idx.gz"

  71. ##

  72. ## $db_annovar_avsift$version_available

  73. ## [1] "avsift"

  74. ##

  75. ## $db_annovar_avsift$version_newest

  76. ## [1] "avsift"

  77. ##

  78. ##

  79. ## $db_annovar_avsnp

  80. ## $db_annovar_avsnp$buildver_available

  81. ## $db_annovar_avsnp$buildver_available$avsnp138

  82. ## [1] "hg19"

  83. ##

  84. ## $db_annovar_avsnp$buildver_available$avsnp142

  85. ## [1] "hg38" "hg19"

  86. ##

  87. ## $db_annovar_avsnp$buildver_available$avsnp144

  88. ## [1] "hg38" "hg19"

  89. ##

  90. ## $db_annovar_avsnp$buildver_available$avsnp147

  91. ## [1] "hg38" "hg19"

  92. ##

  93. ## $db_annovar_avsnp$buildver_available$avsnp150

  94. ## [1] "hg38" "hg19"

  95. ##

  96. ##

  97. ## $db_annovar_avsnp$decompress

  98. ## [1] TRUE TRUE

  99. ##

  100. ## $db_annovar_avsnp$description

  101. ## $db_annovar_avsnp$description$avsnp138

  102. ## [1] "dbSNP138 with allelic splitting and left-normalization"

  103. ##

  104. ## $db_annovar_avsnp$description$avsnp142

  105. ## [1] "dbSNP142 with allelic splitting and left-normalization"

  106. ##

  107. ## $db_annovar_avsnp$description$avsnp144

  108. ## [1] "dbSNP144 with allelic splitting and left-normalization (http://annovar.openbioinformatics.org/en/latest/articles/dbSNP/#additional-discussions)"

  109. ##

  110. ## $db_annovar_avsnp$description$avsnp147

  111. ## [1] "dbSNP147 with allelic splitting and left-normalization"

  112. ##

  113. ##

  114. ## $db_annovar_avsnp$source_url

  115. ## [1] "http://www.openbioinformatics.org/annovar/download/{{buildver}}_{{version}}.txt.gz"    

  116. ## [2] "http://www.openbioinformatics.org/annovar/download/{{buildver}}_{{version}}.txt.idx.gz"

  117. ##

  118. ## $db_annovar_avsnp$version_available

  119. ## [1] "avsnp150" "avsnp147" "avsnp144" "avsnp142" "avsnp138"

  120. ##

  121. ## $db_annovar_avsnp$version_newest

  122. ## [1] "avsnp150"

  123. ##

  124. ##

  125. ## $db_annovar_avsnp_sqlite

  126. ## $db_annovar_avsnp_sqlite$buildver_available

  127. ## [1] "hg19"

  128. ##

  129. ## $db_annovar_avsnp_sqlite$install

  130. ## [1] "#R#sql2sqlite('{{buildver}}_{{version}}.sqlite.sql', sqlite.path = '{{buildver}}_{{version}}.sqlite')#R#"

  131. ##

  132. ## $db_annovar_avsnp_sqlite$source_url

  133. ## [1] "http://bioinfo.rjh.com.cn/download/annovarR/humandb/{{buildver}}_{{version}}.sqlite.sql.gz"

  134. ##

  135. ## $db_annovar_avsnp_sqlite$version_available

  136. ## [1] "avsnp147"        "avsnp147.common" "avsnp144"      

  137. ## [4] "avsnp142"        "avsnp138"      

  138. ##

  139. ## $db_annovar_avsnp_sqlite$version_newest

  140. ## [1] "avsnp147"

  141. ##

  142. ##

  143. ## $title

  144. ## [1] "TOML Example"

配置文件格式转换

  1. # Convert YAML configuration file to JSON format

  2. out.json <- tempfile(fileext = ".json")

  3. convert.config(file = config.yaml, out.file = out.json, convert.to = "JSON")

  1. ## [1] TRUE

  1. get.config.type(out.json)

  1. ## [1] "json"

  1. # Generate a JSON format configuration file

  2. list.test <- list(a = c(123, 456))

  3. out.fn <- sprintf("%s/test.json", tempdir())

  4. write.config(config.dat = list.test, file.path = out.fn, write.type = "json")

  1. ## [1] TRUE

  1. get.config.type(out.fn)

  1. ## [1] "json"

  1. # Generate a YAML format configuration file with defined

  2. # indent

  3. write.config(config.dat = list.test, file.path = out.fn, write.type = "yaml",

  4.  indent = 4)

  1. ## [1] TRUE

  1. get.config.type(out.fn)

  1. ## [1] "yaml"

配置文件扩展解析

为了最大化利用配置文件，我定义了一些规则来进行扩展解析，也就是在jsonlite/ini/yaml/RcppTOML读取配置文件之后进行额外的解析和操作。

  • extra.list可以用来替换配置文件中两个大括号括起来的值，比如 {{debug}}会被替换为 extra.list = list(debug = 'self')中的 self

  • other.config可以被用来联系两个配置文件，如果你设置 other.config =system.file('extdata', 'config.other.yaml', package='configr')，它会在config.other.yaml中读取 key并且获取 yes_flag的值然后替换解析的配置文件中对应的 {{key:yes_flag}}值。

  • rcmd.parse可以被用来解析 @>@str_replace('config','g$','gr')@<@，它可以将这一部分替换为R命令运行的结果。

  • bash.parse可以被用来解析 #>#echo bash#<#，它可以将这一部分替换为系统终端的命令运行结果.

  • glue.parse使用了R包glue进行相关解析，它会替换 !!glue {1:5}变为["1", "2", "3", "4", "5"]; !!glue_numeric {1:5}变为 [1, 2, 3, 4, 5]

下面是一些具体的实例供大家参考。

  1. other.config <- system.file("extdata", "config.other.yaml", package = "configr")

  2. read.config(file = other.config)

  1. ## $key

  2. ## $key$test_parse

  3. ## [1] 123

  4. ##

  5. ## $key$test_parse2

  6. ## [1] 234

  7. ##

  8. ## $key$yes_flag

  9. ## [1] "yes"

  10. ##

  11. ## $key$no_flag

  12. ## [1] "no"

  13. ##

  14. ##

  15. ## $`samtools@1.3.1`

  16. ## $`samtools@1.3.1`$source_dir

  17. ## [1] "/tmp"

  1. config.1 <- read.config(file = config.json)

  2. config.1$default

  1. ## $debug

  2. ## [1] "{{debug}} {{debug2}}"

  1. read.config(file = config.json, extra.list = list(debug = "self",

  2.  debug2 = "self2"))$default

  1. ## $debug

  2. ## [1] "self self2"

  1. sections <- c("default", "other_config_parse")

  2. config.1[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "{{debug}} {{debug2}}"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "{{key:yes_flag}} {{key:no_flag}}"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  1. read.config(file = config.json, extra.list = list(debug = "self",

  2.  debug2 = "self2"), other.config = other.config)[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "self self2"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "yes no"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  1. sections <- c("default", "other_config_parse", "rcmd_parse")

  2. # The followed two line command will return the same value

  3. config.1[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "{{debug}} {{debug2}}"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "{{key:yes_flag}} {{key:no_flag}}"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  12. ##

  13. ##

  14. ## $rcmd_parse

  15. ## $rcmd_parse$raw

  16. ## [1] "@>@ Sys.Date() @<@"

  1. read.config(file = config.json, extra.list = list(debug = "self",

  2.  debug2 = "self2"), other.config = other.config, rcmd.parse = T)[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "self self2"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "yes no"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  12. ##

  13. ##

  14. ## $rcmd_parse

  15. ## $rcmd_parse$raw

  16. ## [1] "2017-11-26"

  1. parse.extra(config.1, extra.list = list(debug = "self", debug2 = "self2"),

  2.  other.config = other.config, rcmd.parse = T)[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "self self2"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "yes no"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  12. ##

  13. ##

  14. ## $rcmd_parse

  15. ## $rcmd_parse$raw

  16. ## [1] "2017-11-26"

  1. sections <- c("default", "other_config_parse", "rcmd_parse",

  2.  "mulitple_parse")

  3. config.1[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "{{debug}} {{debug2}}"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "{{key:yes_flag}} {{key:no_flag}}"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  12. ##

  13. ##

  14. ## $rcmd_parse

  15. ## $rcmd_parse$raw

  16. ## [1] "@>@ Sys.Date() @<@"

  17. ##

  18. ##

  19. ## $mulitple_parse

  20. ## $mulitple_parse$raw

  21. ## [1] "@>@str_replace('config','g$','gr')@<@, #>#echo configr#<#, {{key:yes_flag}}, {{yes}}, @>@str_replace('configr','r','')@<@, #># echo config#<#, {{key:no_flag}}, {{no}}"

  22. ##

  23. ## $mulitple_parse$parsed

  24. ## [1] "configr, configr, yes, 1, config, config, no, 0"

  1. parse.extra(config.1, extra.list = list(debug = "self", debug2 = "self2",

  2.  yes = "1", no = "0"), other.config = other.config, rcmd.parse = T,

  3.  bash.parse = F)[sections]

  1. ## $default

  2. ## $default$debug

  3. ## [1] "self self2"

  4. ##

  5. ##

  6. ## $other_config_parse

  7. ## $other_config_parse$raw

  8. ## [1] "yes no"

  9. ##

  10. ## $other_config_parse$parsed

  11. ## [1] "yes no"

  12. ##

  13. ##

  14. ## $rcmd_parse

  15. ## $rcmd_parse$raw

  16. ## [1] "2017-11-26"

  17. ##

  18. ##

  19. ## $mulitple_parse

  20. ## $mulitple_parse$raw

  21. ## [1] "configr, #>#echo configr#<#, yes, 1, config, #># echo config#<#, no, 0"

  22. ##

  23. ## $mulitple_parse$parsed

  24. ## [1] "configr, configr, yes, 1, config, config, no, 0"

  1. # glue parse

  2. raw <- c("a", "!!glue{1:5}", "c")

  3. list.raw <- list(glue = raw, nochange = 1:10)

  4. list.raw

  1. ## $glue

  2. ## [1] "a"           "!!glue{1:5}" "c"          

  3. ##

  4. ## $nochange

  5. ##  [1]  1  2  3  4  5  6  7  8  9 10

  1. expect.parsed.1 <- c("a", "1", "2", "3", "4", "5", "c")

  2. expect.parsed.2 <- list(glue = expect.parsed.1, nochange = 1:10)

  3. parse.extra(list.raw, glue.parse = TRUE, glue.flag = "!!glue")

  1. ## $glue

  2. ## [1] "a" "1" "2" "3" "4" "5" "c"

  3. ##

  4. ## $nochange

  5. ##  [1]  1  2  3  4  5  6  7  8  9 10

赞 (0)

相关推荐