QNAP TS-212 How to rebuild RAID manually from telnet

Scenario = replace a disk in QNAP TS-212 with RAID 1 configuration active

RAID rebuild should start automatically, but some times it could happen you got stuck with 1 Single Disk + 1 Mirroring Disk Volume:

SingleDisk

According to the QNAP Support – How can I migrate from Single Disk to RAID 0/1 in TS-210/TS-212? , TS-210/TS-212 does not support Online RAID Level Migration. Therefore, please backup the data on the single disk to another location, install the second hard drive, and then recreate the new RAID 0/1 array (hard drive must be formatted).

The workaround is this:

  1. Telnet to NAS as Admin
  2. Check your current disk configuration for Disk #1 and Disk #2 =
    fdisk -l /dev/sda

    Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
    255 heads, 63 sectors/track, 121601 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1               1          66      530125   83  Linux
    /dev/sdb2              67         132      530142   83  Linux
    /dev/sdb3             133      121538   975193693   83  Linux
    /dev/sdb4          121539      121600      498012   83  Linux

    fdisk -l /dev/sdb

    Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
    255 heads, 63 sectors/track, 121601 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1               1          66      530125   83  Linux
    /dev/sda2              67         132      530142   83  Linux
    /dev/sda3             133      121538   975193693   83  Linux
    /dev/sda4          121539      121600      498012   83  Linux

  3. SDA is the first disk, SDB is the second disk
  4. Verify the current status of RAID with this command =
    mdadm –detail /dev/md0

    /dev/md0:
            Version : 00.90.03
      Creation Time : Thu Sep 22 21:50:34 2011
         Raid Level : raid1
         Array Size : 486817600 (464.27 GiB 498.50 GB)
      Used Dev Size : 486817600 (464.27 GiB 498.50 GB)
       Raid Devices : 2
      Total Devices : 1
    Preferred Minor : 0
        Persistence : Superblock is persistent

      Intent Bitmap : Internal

        Update Time : Thu Jul 19 01:13:58 2012
              State : active, degraded
    Active Devices : 1
    Working Devices : 1
    Failed Devices : 0
      Spare Devices : 0

               UUID : 72cc06ac:570e3bf8:427adef1:e13f1b03
             Events : 0.1879365

        Number   Major   Minor   RaidDevice State
           0       0        0        0      removed
           1       8        3        1      active sync  /dev/sda3

  5. As you can see the /dev/sda3 is working, so disk #1 is OK, but disk #2 is missing from RAID
  6. Check if Disk #2 /dev/sdb is mounted (it should be) =
    mount

    /proc on /proc type proc (rw)
    none on /dev/pts type devpts (rw,gid=5,mode=620)
    sysfs on /sys type sysfs (rw)
    tmpfs on /tmp type tmpfs (rw,size=32M)
    none on /proc/bus/usb type usbfs (rw)
    /dev/sda4 on /mnt/ext type ext3 (rw)
    /dev/md9 on /mnt/HDA_ROOT type ext3 (rw)
    /dev/md0 on /share/MD0_DATA type ext4 (rw,usrjquota=aquota.user,jqfmt=vfsv0,user_xattr,data=ordered,delalloc,noacl)
    tmpfs on /var/syslog_maildir type tmpfs (rw,size=8M)
    /dev/sdt1 on /share/external/sdt1 type ufsd (rw,iocharset=utf8,dmask=0000,fmask=0111,force)
    tmpfs on /.eaccelerator.tmp type tmpfs (rw,size=32M)
    /dev/sdb3 on /share/HDB_DATA type ext3 (rw,usrjquota=aquota.user,jqfmt=vfsv0,user_xattr,data=ordered,noacl)

  7. Dismount the /dev/sdb3 Disk #2 with this command =
    umount /dev/sdb3
  8. Add Disk #2 into the RAID /dev/md0 =
    mdadm /dev/md0 –add /dev/sdb3

    mdadm: added /dev/sdb3

  9. Check the RAID status and the rebuild should be started automatically =
    mdadm –detail /dev/md0

    /dev/md0:
            Version : 00.90.03
      Creation Time : Thu Sep 22 21:50:34 2011
         Raid Level : raid1
         Array Size : 486817600 (464.27 GiB 498.50 GB)
      Used Dev Size : 486817600 (464.27 GiB 498.50 GB)
       Raid Devices : 2
      Total Devices : 2
    Preferred Minor : 0
        Persistence : Superblock is persistent

      Intent Bitmap : Internal

        Update Time : Thu Jul 19 01:30:27 2012
              State : active, degraded, recovering
    Active Devices : 1
    Working Devices : 2
    Failed Devices : 0
      Spare Devices : 1

    Rebuild Status : 0% complete

               UUID : 72cc06ac:570e3bf8:427adef1:e13f1b03
             Events : 0.1879848

        Number   Major   Minor   RaidDevice State
           2       8       19        0      spare rebuilding   /dev/sdb3
           1       8        3        1      active sync   /dev/sda3

  10. Check the NAS site for the rebuild % progress

    image

  11. After the RAID rebuild complete, restart NAS to clean all previous mount point folder for sdb3

24 thoughts on “QNAP TS-212 How to rebuild RAID manually from telnet

  1. Thank you for sharing this. I should point that i tried that on my 410 but mdadm command should start by — and not a single one.

    I’m having some trouble trying to replace a failed disk in my raid5 with one of higher capacity (because i haven’t one of same size) rebuild doesn’t start automatically and when i follow your steps i realized there is no sda3 so i’m assuming disk hasn’t been formatted:

    [~] # fdisk -l /dev/sda
    You must set cylinders.
    You can do this from the extra functions menu.

    Disk /dev/sda: 0 MB, 0 bytes
    255 heads, 63 sectors/track, 0 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    /dev/sda1 1 66 530125 83 Linux
    /dev/sda2 67 132 530142 83 Linux

    Please. How can i format the disk, so i can add it to the raid?

  2. After the rebuild and restart of the NAS the “Current Configuration : Logical Volumes” still shows up wrong. Checking via “mdadm –detail /dev/md0” there was gain only one drive active… I will give it a 2nd try. In order to speed up the rebuild I have stopped all service via “/etc/init.d/services.sh stop” – this was speeding up the rebuild by a factor of between 10x and 20x. You can monitor the process via “cat /proc/mdstat”:

    ~] # cat /proc/mdstat
    Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
    md0 : active raid1 sdb3[2] sda3[0]
    1463569600 blocks [2/1] [U_]
    [>………………..] recovery = 2.4% (35956480/1463569600) finish=324.6min speed=73282K/sec
    bitmap: 3/175 pages [12KB], 4096KB chunk

  3. Thanks. It looks like that did the trick. It’s rebuilding right now.

    one remark.

    mdadm –detail /dev/md0

    needed to be:

    mdadm –detail /dev/md0

    and

    mdadm /dev/md0 –add /dev/sdb3

    needed to be:

    mdadm /dev/md0 -–add /dev/sdb3

  4. Pingback: QNAP TS-212 How to rebuild RAID manually from telnet - ROBIT Sistemas Informaticos y Comunicaciones, S.L | ROBIT Sistemas Informaticos y Comunicaciones, S.L

  5. Pingback: QNAP TS-212 How to rebuild RAID manually from telnet - Roberto Martinez Martin - Web Personal | Roberto Martinez Martin – Web Personal

  6. Pingback: Raid Recovery Failed Qnap

  7. Pingback: Qnap Raid Recovery Mdadm

  8. Pingback: Qnap Raid Recovery Ssh

  9. I’m so glad you published this. I had this exact model NAS with this exact problem and it was solved with your steps. I was really getting worried that I wasn’t going to my new disk to work. Whew!

  10. This saved my skin (I think!) – could you post up a screenshot of what volume management looks like after mirroring is complete? My disk #2 still says “single disk drive 2”, whereas number 1 says “Mirroring disk volume drive 1”. Is this correct? The free size is the same on both so I’m guessing it’s ok….

      • Hi Ing – I meant after the mirroring has completed? Mine still says “single disk drive 2” and is not continuing to mirror. For example, if I delete say a 1Gb file, it only removes the 1Gb from disk #1 and is not mirrored by disk 2. Do you have any ideas what could cause this? Thanks for your help! :-)

      • After the rebuild you should not have Single Disk anymore and you should see only a single RAID disk like before. Did you restart your NAS at end of rebuild process?

        Something like this:

      • ok I repeated your instructions and it works perfectly until the final stage with Raid 1 working. However after the reboot it goes back to degraded mode with sdb3 missing from the Raid…. help!

  11. Hi Ing – after the rebuild I have an unmounted single disk 2. Is this wrong? On the ssh terminal using “mdadm –detail /dev/md0”, it appears that everything is ok – status is ‘active’. When I reboot the NAS, disk 2 then gets removed from the RAID again and the same problem happens. Any other ideas?

  12. Hi Marc,
    I faced the problem once… it was because the RAID replicate the data content but the boot partition is only on disk 1.
    Let say we have this scenario
    * disk 1 is broken and replaced with new disk
    * disk 2 is still active

    First time you turn on the NAS, it will boot from disk 2 as disk 1 is empty.
    it will rebuild RAID and copy data on disk 1 but it still using disk 2
    after you reboot, disk 1 will be used to boot instead of disk 2 and that break your raid information.

    Try to do this:
    * Rebuild the RAID to 100%
    * Now, both disk should have all data replicated
    * Shutdown the NAS
    * Unplug the old disk (2 in example above) which was used to boot as it was the only working one.
    * switch new disk (1) to position of old disk (2) so it will be used to boot
    * turn on the NAS and see if it boot correctly
    * check if you have all your data (it should because RAID was rebuilt to 100%)
    * shutdown the NAS
    * connect old disk (2) to the free slot (used previously by disk 1)
    * start NAS and check if all data is there and status or RAID
    * you may need to rebuild RAID once again on old disk (2)

    Summary is:
    * NAS usually boot only from disk 1
    * You need to slip disk 1 and disk 2 to be able to rebuild RAID correctly.

    Hope this help!

  13. Hi!

    Thanks for writing this webpage. It has been very, very detailed and helped me a couple of times.

    We have RAID 1 in two older 2TB drives for the TS-212. However, one drive kept dropping off RAID, so we followed this page and rebuilt the drives sucessfully, and be online for some time (1 day, 1 week, 1 month). Then it will drop out again and run at degraded mode.

    Here is what happened this week

    Type Date Time Users Source IP Computer name Content
    2016/02/08 11:45:15 System 127.0.0.1 localhost [Mirror Disk Volume: Drive 1] RAID device in degraded mode.
    2016/02/08 08:59:37 Mirroring Disk Volume Drive 1 is completed
    2016/02/07 23:42:36 System 127.0.0.1 localhost [Mirror Disk Volume: Drive 1] RAID device in degraded mode.

    This is already the third time we rebuilt it. Has it happened to you or anyone? Thanks in advance for your help.

    Regards,
    Ray

    • Hi Ray,
      I read online that may be caused by several factor.
      If the disk goes to sleep and NAS need to wake it up, if the disk is not fast enough to report back to the controller, it may be marked as bad.
      you should check the SMART count of you HDD and see if you have a high number of recovery count: https://en.wikipedia.org/wiki/S.M.A.R.T.
      Also, in case the disk has some damaged sector, the physical controller try to retry the operation and this cause the NAS controller to mark the disk bad.
      If you want to dig into this topic, look for the difference between RED and GREEN disk on western digital website.
      RED series has Intelligent Error Recovery which try to recovery the bad sector without affecting the response time of the NAS controller.

      If it’s keep going out of sync, probably there may be something wrong with physical disk.

      You may try to download some utility from the manufacture website and perform some test before rebuilding the RAID.

      Thanks,
      Ing. Lele

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s