Linux基础教程：结合awk删除hdfs指定日期前的数据

业务背景
约定五天前的HDFS数据为过期版本数据，写一个awk脚本自动删除过期版本数据$ Hadoop fs -ls /user/pms/workspace/ouyangyewei/data
Found 9 items
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-01
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-02
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-03
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-04
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-05
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-06
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-07
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-08
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-09
脚本实现
# ---------------------------------------------------------
#
# 删除历史版本（五天前的为过期版本数据）
#
# ---------------------------------------------------------old_version=$（hadoop fs -ls /user/pms/workspace/ouyangyewei/data | awk "BEGIN{ five_days_ago=strftime（"%F", systime（）-5*24*3600） }{ split（$8,arr,"/"）; if（arr[7]<five_days_ago）{printf "%s ", $8} }"）
arr=（${old_version// / }）
for version in ${arr[@]}
do
hadoop fs -rmr $version
done执行以后$ hadoop fs -ls /user/pms/workspace/ouyangyewei/data
Found 4 items
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-06
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-07
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-08
drwxr-xr-x - pms pms 0 2015-08-11 17:03 /user/pms/workspace/ouyangyewei/data/2015-08-09awk赋值操作符 http://www.linuxidc.com/Linux/2015-06/118887.htmawk转义序列与算术操作符 http://www.linuxidc.com/Linux/2015-06/118886.htmAWK简介及使用实例 http://www.linuxidc.com/Linux/2013-12/93519.htmAWK 简介和例子 http://www.linuxidc.com/Linux/2012-12/75441.htmShell脚本之AWK文本编辑器语法 http://www.linuxidc.com/Linux/2013-11/92787.htm正则表达式中AWK的学习和使用 http://www.linuxidc.com/Linux/2013-10/91892.htm文本数据处理之AWK 图解 http://www.linuxidc.com/Linux/2013-09/89589.htm如何在Linux中使用awk命令 http://www.linuxidc.com/Linux/2014-10/107542.htm文本分析工具-awk http://www.linuxidc.com/Linux/2014-12/110939.htm本文永久更新链接地址