Table of Contents

Introduction

In today’s how-to guide, let’s walk through How to Recover a deleted logical volume in logical volume management (LVM). To perform the restore we will use vgcfgrestore command.

While performing any logical volume management task, In case, if you have accidentally deleted the logical volume it can be restored with its data. In the following step, will show how to recover a deleted logical volume by removing anyone of LV.

If you are looking to create a logical volume or advance logical volume topics, have a look at this.

Logical Volume Management guides.

Existing FileSystems

These are the filesystem we have in our server. To perform this demonstration we have created these two filesystems and copied a few files.

[root@prod-srv-01 ~]# df -hP /data/ /u01/
 Filesystem                   Size  Used Avail Use% Mounted on
 /dev/mapper/vg_data-lv_data   25G   24G  1.2G  96% /data
 /dev/mapper/vg_data-lv_u01    25G  6.4G   19G  26% /u01
[root@prod-srv-01 ~]#

Few of logs and oracle binary files are under /u01 mount point, and some of backups are stored in /data mount points.

[root@prod-srv-01 ~]# ls -lthr /u01/
 total 6.2G
 -rw-r--r--. 1 root root 200M Dec 14 00:10 app.log
 -rw-r--r--. 1 root root  20M Dec 14 00:10 oracle_error.log
 -rw-r--r--. 1 root root 1.5G Dec 14 00:12 linux.x64_11gR2_database_1of2.zip
 -rw-r--r--. 1 root root 1.0G Dec 14 00:12 linux.x64_11gR2_database_2of2.zip
 -rw-r--r--. 1 root root 2.0G Dec 14 00:13 linuxamd64_12102_database_1of2.zip
 -rw-r--r--. 1 root root 1.5G Dec 14 00:13 linuxamd64_12102_database_2of2.zip
[root@prod-srv-01 ~]# 

[root@prod-srv-01 ~]# ls -lthr /data/
 total 19G
 -rw-r--r--. 1 root root 2.0G Dec 13 23:57 web_mysql_db_backup.sql
 drwxr-xr-x. 2 root root   86 Dec 13 23:58 backups
 -rw-r--r--. 1 root root 5.0M Dec 14 00:00 backups.tar.gz
 drwxr-xr-x. 2 root root   66 Dec 14 00:03 playbooks
[root@prod-srv-01 ~]#

Physical Volume used Under LV

Let’s show the list of physical volumes used under the lv_u01 logical volume.

[root@prod-srv-01 ~]# lvs -o +devices /dev/mapper/vg_data-lv_u01 
   LV     VG      Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices       
   lv_u01 vg_data -wi-ao---- 25.00g                                                     /dev/sdd(0)   
   lv_u01 vg_data -wi-ao---- 25.00g                                                     /dev/sdc(1281)
[root@prod-srv-01 ~]#

Accidentally Deleted Logical Volume

For demonstration purpose lets remove the lv_u01 logical volume.

[root@prod-srv-01 ~]# lvremove /dev/mapper/vg_data-lv_u01 
 Do you really want to remove active logical volume vg_data/lv_u01? [y/n]: y
   Logical volume "lv_u01" successfully removed
[root@prod-srv-01 ~]#

Print and Confirm using lvm that we have removed the logical volume.

[root@prod-srv-01 ~]# lvs
   LV      VG      Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
   home    rhel    -wi-ao---- 30.98g                                                    
   root    rhel    -wi-ao---- 63.46g                                                    
   swap    rhel    -wi-ao----  3.96g                                                    
   lv_data vg_data -wi-ao---- 25.00g                                                    
[root@prod-srv-01 ~]#

Since the lv removed we are able to see 35 GB of free space in Volume group.

[root@prod-srv-01 ~]# vgs
   VG      #PV #LV #SN Attr   VSize   VFree  
   rhel      1   3   0 wz--n-  98.41g      0 
   vg_data   3   1   0 wz--n- <59.99g <34.99g
[root@prod-srv-01 ~]#

Begin with Restore

Now let’s see how to restore the deleted logical volume. The two directories we are interested are

[root@prod-srv-01 ~]# ls -lthr /etc/lvm/backup/
 total 8.0K
 -rw-------. 1 root root 2.1K Dec 12 00:26 rhel
 -rw-------. 1 root root 2.0K Dec 14 00:16 vg_data
[root@prod-srv-01 ~]#

and

[root@prod-srv-01 ~]# ls -lthr /etc/lvm/archive/
 total 20K
 -rw-------. 1 root root 2.1K Dec 12 00:26 rhel_00000-2091684635.vg
 -rw-------. 1 root root 1.4K Dec 13 23:51 vg_data_00000-483730858.vg
 -rw-------. 1 root root 1.4K Dec 13 23:52 vg_data_00001-334365735.vg
 -rw-------. 1 root root 2.0K Dec 13 23:52 vg_data_00002-1010471410.vg
 -rw-------. 1 root root 2.5K Dec 14 00:16 vg_data_00003-528295817.vg
[root@prod-srv-01 ~]#

You may ask. So, how these backups and archive metadata’s are created? And when they are created?

Whenever, we perform any tasks related to LVM like disk extend, delete or add everything will be recorded and stored under /etc/lvm/archive. In meantime, it will take a backup of VG configuration with the help of vgcfgbackup command and store it under /etc/lvm/backup.

If you would have used --verbose or -v option while removing the lv we could see the actual steps happen behind the scenes.

[root@prod-srv-01 ~]# lvremove -v /dev/mapper/vg_data-lv_u01 
 Do you really want to remove active logical volume vg_data/lv_u01? [y/n]: y
   Accepted input: [y]
   Removing vg_data-lv_u01 (253:4)
   Archiving volume group "vg_data" metadata (seqno 4).
   Releasing logical volume "lv_u01"
   Creating volume group backup "/etc/lvm/backup/vg_data" (seqno 5).
   Logical volume "lv_u01" successfully removed
[root@prod-srv-01 ~]#

Beginning with restoring, lets list and check how many VG metadata backups are available.

# vgcfgrestore ---list vg_data

--list – List metadata backup and archive files pertaining to the VG.

[root@prod-srv-01 ~]# vgcfgrestore ---list vg_data
 File:        /etc/lvm/archive/vg_data_00000-483730858.vg
   VG name:        vg_data
   Description:    Created before executing 'vgcreate vg_data /dev/sdb /dev/sdc /dev/sdd'
   Backup Time:    Sun Dec 13 23:51:08 2020
 File:        /etc/lvm/archive/vg_data_00001-334365735.vg
   VG name:        vg_data
   Description:    Created before executing 'lvcreate -L +25G -n lv_data vg_data'
   Backup Time:    Sun Dec 13 23:52:35 2020
 File:        /etc/lvm/archive/vg_data_00002-1010471410.vg
   VG name:        vg_data
   Description:    Created before executing 'lvcreate -L +25G -n lv_u01 vg_data'
   Backup Time:    Sun Dec 13 23:52:52 2020
 File:        /etc/lvm/archive/vg_data_00003-528295817.vg
   VG name:        vg_data
   Description:    Created before executing 'lvremove /dev/mapper/vg_data-lv_u01'
   Backup Time:    Mon Dec 14 00:16:02 2020
 File:        /etc/lvm/backup/vg_data
   VG name:        vg_data
   Description:    Created after executing 'lvremove /dev/mapper/vg_data-lv_u01'
   Backup Time:    Mon Dec 14 00:16:02 2020
[root@prod-srv-01 ~]#

To do a dry run use --test with vg name by pointing to the archive

[root@prod-srv-01 ~]# vgcfgrestore vg_data --test -f /etc/lvm/archive/vg_data_00003-528295817.vg 
   TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.
   Volume group vg_data has active volume: lv_data.
   WARNING: Found 1 active volume(s) in volume group "vg_data".
   Restoring VG with active LVs, may cause mismatch with its metadata.
 Do you really want to proceed with restore of volume group "vg_data", while 1 volume(s) are active? [y/n]: y
   Restored volume group vg_data.
[root@prod-srv-01 ~]#

Remove the --test and start the restore.

[root@prod-srv-01 ~]# vgcfgrestore vg_data -f /etc/lvm/archive/vg_data_00003-528295817.vg 
   Volume group vg_data has active volume: lv_data.
   WARNING: Found 1 active volume(s) in volume group "vg_data".
   Restoring VG with active LVs, may cause mismatch with its metadata.
 Do you really want to proceed with restore of volume group "vg_data", while 1 volume(s) are active? [y/n]: y
   Restored volume group vg_data.
[root@prod-srv-01 ~]#

-f – Read anyone of metadata file created by vgcfgbackup.

We are able to see the lv_u01 is restored and in inactive state.

[root@prod-srv-01 ~]# lvscan 
   ACTIVE            '/dev/vg_data/lv_data' [25.00 GiB] inherit
   inactive          '/dev/vg_data/lv_u01' [25.00 GiB] inherit
   ACTIVE            '/dev/rhel/swap' [3.96 GiB] inherit
   ACTIVE            '/dev/rhel/home' [30.98 GiB] inherit
   ACTIVE            '/dev/rhel/root' [63.46 GiB] inherit
[root@prod-srv-01 ~]#

Activating Logical Volume

Now, we can change the lv from inactive to active mode.

# lvchange -a y /dev/vg_data/lv_u01

-a will activate the logical volume
-y represent “yes”

Once again verify, we could see its in active state.

[root@prod-srv-01 ~]# lvscan 
   ACTIVE            '/dev/vg_data/lv_data' [25.00 GiB] inherit
   ACTIVE            '/dev/vg_data/lv_u01' [25.00 GiB] inherit
   ACTIVE            '/dev/rhel/swap' [3.96 GiB] inherit
   ACTIVE            '/dev/rhel/home' [30.98 GiB] inherit
   ACTIVE            '/dev/rhel/root' [63.46 GiB] inherit
[root@prod-srv-01 ~]#

Mount and Verify

After activating the LV it should automatically mount the filesystem. Else, mount by running # mount /dev/mapper/vg_data-lv_u01 /u01/.

[root@prod-srv-01 ~]# df -hP /u01/
 Filesystem                  Size  Used Avail Use% Mounted on
 /dev/mapper/vg_data-lv_u01   25G  6.4G   19G  26% /u01
[root@prod-srv-01 ~]#

We are good, still we have our files.

[root@prod-srv-01 ~]# ls -lthr /u01/
 total 6.2G
 -rw-r--r--. 1 root root 200M Dec 14 00:10 app.log
 -rw-r--r--. 1 root root  20M Dec 14 00:10 oracle_error.log
 -rw-r--r--. 1 root root 1.5G Dec 14 00:12 linux.x64_11gR2_database_1of2.zip
 -rw-r--r--. 1 root root 1.0G Dec 14 00:12 linux.x64_11gR2_database_2of2.zip
 -rw-r--r--. 1 root root 2.0G Dec 14 00:13 linuxamd64_12102_database_1of2.zip
 -rw-r--r--. 1 root root 1.5G Dec 14 00:13 linuxamd64_12102_database_2of2.zip
[root@prod-srv-01 ~]#

That’s it, we have completed with restoring a deleted logical volume in LVM.

Conclusion

We have managed to restore the deleted logical volume from metadata of a Volume group. In upcoming guide, let see how to restore a PV. Subscribe to our newsletter and receive our regular updates.

jos on How to Configure Sudo email alertMarch 4, 2022
Hi! Some way to send commands write as root by email? Thanks!
paul on Monitoring Proxmox with InfluxDB and Grafana in 4 Easy stepsMarch 1, 2022
Tried 12966 per your example, got error "failed to import dashboard."
Anonymous on Migrate existing Iptables to Nftables in RHEL8/CentOSFebruary 27, 2022
I followed the procedures and successfully loaded some weird 8081 port configurations needed for the app, however I discovered it…
sy on Install Ansible AWX on Kubernetes in 5 minutesFebruary 17, 2022
tried many times - followed all steps but was unable to setup custom context I always get -> The connection…
Faraz Ahmed on Prometheus with Grafana for Linux server monitoring in 5 easy stepsFebruary 7, 2022
Hello Babin Lonston, Your tutorial is very helpful to me to Install the prometheus and nodeexporter on the server. Can…