一次ubuntu系统硬盘read-only file system的处理过程

当前位置：首页 > 系统 > 正文

Mar102014

作者：Fisher 发布：2014-03-10 21:15 分类：系统 1条评论

［摘要］一台运行了n年的samba服务器硬盘出现了read-only file system，里面的数据非常重要，不过之前有备份，所以还是比较敢于折腾，不过还是几经周折才最终搞定，下面把整个处理过程总结记录一下，如能帮助到其他人，那更是乐意至极。

症状:
read-only file system，不过数据还是可以备份出来：

$touch sudops.com
Read-only file system

1 2	$touch sudops.com Read-only file system

操作系统：

Ubuntu 9.10 \n \l

1	Ubuntu 9.10 \n \l

内核版本：

Linux localhost 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 01:26:53 UTC 2010 i686 GNU/Linux

1	Linux localhost 2.6.31-19-generic #56-Ubuntu SMP Thu Jan 28 01:26:53 UTC 2010 i686 GNU/Linux

文件系统：
reiserfs

处理过程：
1. 首先用rsync进行了一次完全备份，因为不确定之前的备份是否完整。
2. 使用reiserfsck检查一下有问题的分区：

reiserfsck --check /dev/sda7

1	reiserfsck --check /dev/sda7

…一堆错误，还好貌似能够修复。

bad_path: block 13108014, pointer 93: The used space (3948) of the child block (11058482) is not equal to the (blocksize (4096) - fr/ 14 (of  14\/ 10 (of 142-/131 (of 170/bad_stat_data: The objectid (3006112) is shared by at least two files. Can be fixed with --rebuild-tree only.                                    / 11 (of 142// 15 (of  86|bad_stat_data: The objectid (3006142) is shared by at least two files. Can be fixed with --rebuild-tree only.                                    / 13 (of 142-/102 (of 170|bad_stat_data: The objectid (3006845) is shared by at least two files. Can be fixed with --rebuild-tree only.                      finished                                
Comparing bitmaps..finished
Fatal corruptions were found, Semantic pass skipped
1 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Mon Mar 10 18:49:01 2014
###########

bad_path: block 13108014, pointer 93: The used space (3948) of the child block (11058482) is not equal to the (blocksize (4096) - fr/ 14 (of 14\/ 10 (of 142-/131 (of 170/bad_stat_data: The objectid (3006112) is shared by at least two files. Can be fixed with --rebuild-tree only. / 11 (of 142// 15 (of 86|bad_stat_data: The objectid (3006142) is shared by at least two files. Can be fixed with --rebuild-tree only. / 13 (of 142-/102 (of 170|bad_stat_data: The objectid (3006845) is shared by at least two files. Can be fixed with --rebuild-tree only. finished

Comparing bitmaps..finished

Fatal corruptions were found, Semantic pass skipped

1 found corruptions can be fixed only when running with --rebuild-tree

###########

reiserfsck finished at Mon Mar 10 18:49:01 2014

###########

3. 使用reiserfsck修复磁盘分区，用到了rebuild-tree。

reiserfsck --scan-whole-partition --rebuild-tree /dev/sda7

1	reiserfsck --scan-whole-partition --rebuild-tree /dev/sda7

命令执行了很长时间，强烈建议在screen中运行。

reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************
** Do not  run  the  program  with  --rebuild-tree  unless **
** something is broken and MAKE A BACKUP  before using it. **
** If you have bad sectors on a drive  it is usually a bad **
** idea to continue using it. Then you probably should get **
** a working hard drive, copy the file system from the bad **
** drive  to the good one -- dd_rescue is  a good tool for **
** that -- and only then run this program.                 **
** If you are using the latest reiserfsprogs and  it fails **
** please  email bug reports to reiserfs-list@namesys.com, **
** providing  as  much  information  as  possible --  your **
** hardware,  kernel,  patches,  settings,  all reiserfsck **
** messages  (including version),  the reiserfsck logfile, **
** check  the  syslog file  for  any  related information. **
** If you would like advice on using this program, support **
** is available  for $25 at  www.namesys.com/support.html. **
*************************************************************

Will rebuild the filesystem (/dev/sda7) tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes
Replaying journal: Done.
Reiserfs journal '/dev/sda7' in blocks [18..8211]: 0 transactions replayed
###########
reiserfsck --rebuild-tree started at Mon Mar 10 18:49:34 2014
###########

Pass 0:
####### Pass 0 #######
The whole partition (24999136 blocks) is to be scanned
Skipping 8973 blocks (super block, journal, bitmaps) 24990163 blocks will be read
0%...block 3968588: The number of items (6144) is incorrect, should be (1) - corrected
block 3968588: The free space (16128) is incorrect, should be (4048) - corrected
.20%...block 8115050: The number of items (15) is incorrect, should be (1) - corrected
block 8115050: The free space (0) is incorrect, should be (3792) - corrected
pass0: vpf-10110: block 8115050, item (0): Unknown item type found [4608 16778752 0x48000000 ??? (15)] - deleted
.40%...block 13045490: The number of items (6144) is incorrect, should be (1) - corrected
block 13045490: The free space (16128) is incorrect, should be (4048) - corrected
.60%..pass0: vpf-10140: block 17696155: items 5 and 6 have bad short keys [2854916 2854916 0x0 SD (0)], [2854916 2854916 0x1 IND (1)], both deleted
..80%.block 21445883: The number of items (33792) is incorrect, should be (1) - corrected
block 21445883: The free space (65055) is incorrect, should be (4048) - corrected
pass0: vpf-10110: block 21445883, item (0): Unknown item type found [69272617 134288449 0x10002 ??? (15)] - deleted
...100%                       left 0, 23420 /sec
1825888 directory entries were hashed with "r5" hash.
        "r5" hash is selected
Flushing..finished
        Read blocks (but not data blocks) 24990163
                Leaves among those 325186
                        - leaves all contents of which could not be saved and deleted 4
                Objectids found 1854744

Pass 1 (will try to insert 325182 leaves):
####### Pass 1 #######
Looking for allocable blocks .. finished
0%....20%....40%                                         left 183038, 1465 /sec
....60%....80%....100%                        left 0, 1178 /sec
Flushing..finished
        325182 leaves read
                316760 inserted
                        - pointers in indirect items pointing to metadata 442 (zeroed)
                8422 not inserted
        non-unique pointers in indirect items (zeroed) 217907
####### Pass 2 #######

Pass 2:
0%...rewrite_file: 1 items of file [3025169 3025171] moved to [3025169 157]/sec
.20%....40%rewrite_file: 1 items of file [3028211 3028212] moved to [3028211 245]
..vpf-10260: The file we are inserting the new item (66606 1986481 0x621 DRCT (2), len 624, location 3472 entry count 65535, fsck need 0, format new) into has no StatData, insertion was skipped


100%                         left 0, 285 /sec
Flushing..finished
        Leaves inserted item by item 8422
Pass 3 (semantic):
####### Pass 3 #########

Flushing..finished
        Objects without names 35413
        Empty lost dirs removed 2875
        Dirs linked to /lost+found: 4684
                Dirs without stat data found 301
        Files linked to /lost+found 30729
        Objects having used objectids: 5816
                files fixed 5728
                dirs fixed 88
Pass 4 - finisheddone 325462, 178 /sec
        Deleted unreachable items 4736
Flushing..finished
Syncing..finished
###########
reiserfsck finished at Mon Mar 10 19:42:27 2014
###########

reiserfsck 3.6.21 (2009 www.namesys.com)

*************************************************************

** Do not run the program with --rebuild-tree unless **

** something is broken and MAKE A BACKUP before using it. **

** If you have bad sectors on a drive it is usually a bad **

** idea to continue using it. Then you probably should get **

** a working hard drive, copy the file system from the bad **

** drive to the good one -- dd_rescue is a good tool for **

** that -- and only then run this program. **

** If you are using the latest reiserfsprogs and it fails **

** please email bug reports to reiserfs-list@namesys.com, **

** providing as much information as possible -- your **

** hardware, kernel, patches, settings, all reiserfsck **

** messages (including version), the reiserfsck logfile, **

** check the syslog file for any related information. **

** If you would like advice on using this program, support **

** is available for $25 at www.namesys.com/support.html. **

*************************************************************

Will rebuild the filesystem (/dev/sda7) tree

Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes

Replaying journal: Done.

Reiserfs journal '/dev/sda7' in blocks [18..8211]: 0 transactions replayed

###########

reiserfsck --rebuild-tree started at Mon Mar 10 18:49:34 2014

###########

Pass 0:

####### Pass 0 #######

The whole partition (24999136 blocks) is to be scanned

Skipping 8973 blocks (super block, journal, bitmaps) 24990163 blocks will be read

0%...block 3968588: The number of items (6144) is incorrect, should be (1) - corrected

block 3968588: The free space (16128) is incorrect, should be (4048) - corrected

.20%...block 8115050: The number of items (15) is incorrect, should be (1) - corrected

block 8115050: The free space (0) is incorrect, should be (3792) - corrected

pass0: vpf-10110: block 8115050, item (0): Unknown item type found [4608 16778752 0x48000000 ??? (15)] - deleted

.40%...block 13045490: The number of items (6144) is incorrect, should be (1) - corrected

block 13045490: The free space (16128) is incorrect, should be (4048) - corrected

.60%..pass0: vpf-10140: block 17696155: items 5 and 6 have bad short keys [2854916 2854916 0x0 SD (0)], [2854916 2854916 0x1 IND (1)], both deleted

..80%.block 21445883: The number of items (33792) is incorrect, should be (1) - corrected

block 21445883: The free space (65055) is incorrect, should be (4048) - corrected

pass0: vpf-10110: block 21445883, item (0): Unknown item type found [69272617 134288449 0x10002 ??? (15)] - deleted

...100% left 0, 23420 /sec

1825888 directory entries were hashed with "r5" hash.

"r5" hash is selected

Flushing..finished

Read blocks (but not data blocks) 24990163

Leaves among those 325186

- leaves all contents of which could not be saved and deleted 4

Objectids found 1854744

Pass 1 (will try to insert 325182 leaves):

####### Pass 1 #######

Looking for allocable blocks .. finished

0%....20%....40% left 183038, 1465 /sec

....60%....80%....100% left 0, 1178 /sec

Flushing..finished

325182 leaves read

316760 inserted

- pointers in indirect items pointing to metadata 442 (zeroed)

8422 not inserted

non-unique pointers in indirect items (zeroed) 217907

####### Pass 2 #######

Pass 2:

0%...rewrite_file: 1 items of file [3025169 3025171] moved to [3025169 157]/sec

.20%....40%rewrite_file: 1 items of file [3028211 3028212] moved to [3028211 245]

..vpf-10260: The file we are inserting the new item (66606 1986481 0x621 DRCT (2), len 624, location 3472 entry count 65535, fsck need 0, format new) into has no StatData, insertion was skipped

100% left 0, 285 /sec

Flushing..finished

Leaves inserted item by item 8422

Pass 3 (semantic):

####### Pass 3 #########

Flushing..finished

Objects without names 35413

Empty lost dirs removed 2875

Dirs linked to /lost+found: 4684

Dirs without stat data found 301

Files linked to /lost+found 30729

Objects having used objectids: 5816

files fixed 5728

dirs fixed 88

Pass 4 - finisheddone 325462, 178 /sec

Deleted unreachable items 4736

Flushing..finished

Syncing..finished

###########

reiserfsck finished at Mon Mar 10 19:42:27 2014

###########

看上去果然修复成功了，庆祝一下。

4. 重新mount分区

#mount -a （或者: mount -t reiserfs /dev/sda7 /home）
mount: /dev/sda7 已被挂载或 /home 正忙

fuser -m /dev/sda7
lsof /dev/sda7

#mount -a （或者: mount -t reiserfs /dev/sda7 /home）

mount: /dev/sda7 已被挂载或 /home 正忙

fuser -m /dev/sda7

lsof /dev/sda7

都查过了，并没有发现有人使用/home分区

折腾了半天最后发现是有另外一个screen在/home目录里面，汗一个。退出该screen，再进行mount，这下成功了。

检查一下：

df -h
/dev/sda7              96G   85G   11G  89% /home

touch aa 
可写，成功了！

df -h

/dev/sda7 96G 85G 11G 89% /home

touch aa

可写，成功了！

总结一下：磁盘出现read-only没必要慌张，能备份一定要先备份，然后使用fsck等工具尝试修复磁盘。

本文固定链接: https://www.sudops.com/resolve-ubuntu-disk-read-only.html | 运维·速度

该日志由 Fisher 于2014年03月10日发表在系统分类下，你可以发表评论，并在保留原文地址及作者的情况下引用到你的网站或博客。
原创文章转载请注明: 一次ubuntu系统硬盘read-only file system的处理过程 | 运维·速度
关键字: reiserfs

【上一篇】Apache Spark（火花）成为ASF的顶级项目
【下一篇】keepalived＋redis实现HA的验证

您可能还会对这些文章感兴趣！