Tag - Ubuntu

Linux LVM-based Multiple Disks System/Data Migration

What I Have

  1. Current Setup

    • Disk /dev/nvme0n1:
      • Partition 1 (/dev/nvme0n1p1): EFI System, mounted as /boot/efi.
      • Partition 2 (/dev/nvme0n1p2): Linux FileSystem, mounted as /boot.
      • Partition 3 (/dev/nvme0n1p3): LVM, part of ubuntu-vg Volume Group, mounted as /.
    • Disk /dev/nvme2n1:
      • Entire disk is an LVM Physical Volume, also part of ubuntu-vg.

    Output of sudo pvs:

    PV             VG        Fmt  Attr PSize    PFree
/dev/nvme0n1p3 ubuntu-vg lvm2 a--  <473.89g    0
/dev/nvme2n1   ubuntu-vg lvm2 a--  <476.94g    0
  2. New Disk /dev/nvme1n1:
    • Empty, unformatted disk.
    • To be partitioned and used to replace /dev/nvme0n1 and /dev/nvme2n1.

What I Want

  1. Disk /dev/nvme1n1 Setup:
    • Partition 1 (/dev/nvme1n1p1): EFI System, 1 GiB, formatted as vfat, mounted as /boot/efi.
    • Partition 2 (/dev/nvme1n1p2): Linux FileSystem, 2 GiB, formatted as ext4, mounted as /boot.
    • Partition 3 (/dev/nvme1n1p3): LVM, occupies the remaining space, part of ubuntu-vg, mounted as /.
  2. Complete Migration:
    • All data (including /boot, /boot/efi, and /) migrated to /dev/nvme1n1.
    • GRUB configured to boot from /dev/nvme1n1.
    • /dev/nvme0n1 and /dev/nvme2n1 removed after migration.

What I Do

Step 1: Partition the New Disk

  1. Launch fdisk to partition /dev/nvme1n1:
    sudo fdisk /dev/nvme1n1
  2. Create partitions:
    • Partition 1:
      • Size: 1 GiB.
      • Type: EFI System (t, code 1).
    • Partition 2:
      • Size: 2 GiB.
      • Type: Linux FileSystem.
    • Partition 3:
      • Size: Remaining space.
      • Type: Linux LVM (t, code 8e).
  3. Save and exit fdisk (w).
  4. Format the partitions:
    sudo mkfs.vfat -F32 /dev/nvme1n1p1
sudo mkfs.ext4 /dev/nvme1n1p2
sudo pvcreate /dev/nvme1n1p3
  5. Add /dev/nvme1n1p3 to the volume group:
    sudo vgextend ubuntu-vg /dev/nvme1n1p3

Step 2: Migrate LVM Data

  1. Move LVM data from old disks to the new disk:
    sudo pvmove /dev/nvme0n1p3 /dev/nvme1n1p3
sudo pvmove /dev/nvme2n1 /dev/nvme1n1p3
  2. Remove old Physical Volumes:
    sudo vgreduce ubuntu-vg /dev/nvme0n1p3
sudo vgreduce ubuntu-vg /dev/nvme2n1
sudo pvremove /dev/nvme0n1p3
sudo pvremove /dev/nvme2n1

Step 3: Migrate Boot Data

  1. Mount new partitions
    sudo mount /dev/nvme1n1p2 /mnt/target_p2
sudo mkdir -p /mnt/target_p2/boot/efi
sudo mount /dev/nvme1n1p1 /mnt/target_p2/boot/efi
  2. Copy data from old /boot and /boot/efi:
    sudo rsync -avh /boot/ /mnt/target_p2/
sudo rsync -avh /boot/efi/ /mnt/target_p2/boot/efi/

Step 4: Update /etc/fstab

  1. Mount the new root Logical Volume:
    sudo mount /dev/mapper/ubuntu--vg-root /mnt/target_root
  2. Bind /boot and /boot/efi to the new root:
    sudo mount --bind /mnt/target_p2 /mnt/target_root/boot
sudo mount --bind /mnt/target_p2/boot/efi /mnt/target_root/boot/efi
  3. Edit /mnt/target_root/etc/fstab:
    • Update UUIDs for /boot/efi, /boot, and / using blkid:
      sudo blkid
      Example /etc/fstab:
      UUID=<new-efi-uuid> /boot/efi vfat defaults 0 1
UUID=<new-boot-uuid> /boot ext4 defaults 0 2

Step 5: Install GRUB

  1. Enter a chroot environment:
    sudo mount --bind /dev /mnt/target_root/dev
sudo mount --bind /proc /mnt/target_root/proc
sudo mount --bind /sys /mnt/target_root/sys
sudo mount --bind /sys/firmware/efi/efivars /mnt/target_root/sys/firmware/efi/efivars
sudo chroot /mnt/target_root
  2. Install GRUB:
    grub-install
update-grub
  3. Exit chroot:
    exit

Step 6: Verify and Test

  1. Set /dev/nvme1n1 as the boot device in your BIOS/UEFI settings.
  2. Remove /dev/nvme0n1 and /dev/nvme2n1 from the machine.
  3. Boot into the system and verify:
    lsblk
cat /etc/fstab
df -h

Step 7: Resize the New Volume

  1. Extend the LV:
    sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
  2. Resize the Filesystem
    sudo resize2fs /dev/ubuntu-vg/ubuntu-lv

- DONE -

migege posted @ 2024-12-10 11:05:44 358 views 0 comments Ubuntu Linux LVM

Ubuntu Server 22.04.2 LTS 启动卡在 A start job is running for wait for network to be configured

解决方法

sudo systemctl disable systemd-networkd-wait-online.service
sudo systemctl mask systemd-networkd-wait-online.service

ref

migege posted @ 2023-04-07 17:26:47 1626 views 0 comments Ubuntu

TensorFlow 实战 01：安装 GPU 版本的开发环境 (Ubuntu)

这里将介绍如何在 Ubuntu 16.04 LTS 系统上搭建 支持 GPU 的 TensorFlow 1.4.0 开发环境。

是否需要 GPU 支持？

这取决于你有没有一块儿支持 CUDA 的 NVIDIA 显卡。如果没有，只能选择 CPU 版本。如果有，继续往下看。

安装 NVIDIA 依赖

  1. 安装 CUDA Toolkit 9.0：在 CUDA Downloads 页面选择操作系统及版本，安装类型选择deb (network)，最后会给出一个下载链接和一系列的命令，类似：
    sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda
    修改 PATH 环境变量
    export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
    修改 LD_LIBRARY_PATH 环境变量
    export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
  2. 安装显卡驱动，目前最新的驱动版本是 384
    sudo apt install nvidia-384
    驱动安装成功后，可以使用下面的命令查看显卡状态：
    nvidia-smi
  3. 安装 cuDNN 7，在 cuDNN下载页面 点击 Download 并填写调查问卷后，根据自己的系统环境下载对应的安装包并安装，以下是 64 位系统的示例：
    sudo dpkg -i libcudnn7_7.0.3.11-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.0.3.11-1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-doc_7.0.3.11-1+cuda9.0_amd64.deb
  4. 安装 libcupti-dev库
    sudo apt install libcupti-dev
    修改 LD_LIBRARY_PATH 环境变量
    export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH

安装 TensorFlow

安装预编译的包或者从源码编译都是可行的。

原生 pip 安装

python 2.7/3.n 都可以。

  1. 先安装并升级 pip
    # for Python 2.7
sudo apt-get install python-pip python-dev
sudo pip install -U pip setuptools
# for Python 3.n
sudo apt-get install python3-pip python3-dev
sudo pip3 install -U pip setuptools
  2. 安装 tensorflow，根据需求只执行一条命令即可
    pip install tensorflow      # Python 2.7; CPU support (no GPU support)
pip3 install tensorflow     # Python 3.n; CPU support (no GPU support)
pip install tensorflow-gpu  # Python 2.7; GPU support
pip3 install tensorflow-gpu # Python 3.n; GPU support

源码编译安装

  1. git 下载源码仓库，切到 r1.4 分支
    git clone https://github.com/tensorflow/tensorflow
git checkout r1.4
  2. 安装 bazel
    sudo apt-get install openjdk-8-jdk
echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
sudo apt-get update && sudo apt-get install bazel
  3. 安装 TensorFlow 的 Python 依赖
    sudo apt-get install python-numpy python-dev python-pip python-wheel     # for Python 2.7
sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel # for Python 3.n
  4. 执行安装配置，务必注意每一步的选择
    cd tensorflow # 进入第 1 步克隆的仓库根目录
./configure
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7
Found possible Python library paths:
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python2.7/dist-packages]
Using python library path: /usr/local/lib/python2.7/dist-packages
Do you wish to build TensorFlow with MKL support? [y/N]
No MKL support will be enabled for TensorFlow
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? [Y/n]
jemalloc enabled
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Do you wish to build TensorFlow with VERBS support? [y/N]
No VERBS support will be enabled for TensorFlow
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N] Y
CUDA support will be enabled for TensorFlow
Do you want to use clang as CUDA compiler? [y/N]
nvcc will be used as CUDA compiler
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 8.0]: 9.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 6.0]: 7
Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: "3.5,5.2"]: 6.1
Do you wish to build TensorFlow with MPI support? [y/N] 
MPI support will not be enabled for TensorFlow
Configuration finished
  5. 编译生成 pip 包
    bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
  6. 安装生成好的 pip 包，具体的 whl 包在 /tmp/tensorflow_pkg 目录下，文件名可能略有不同
    sudo pip install /tmp/tensorflow_pkg/tensorflow-1.4.0-cp27-cp27mu-linux_x86_64.whl

验证一下是否装成功了

  1. 启动 Python
    $ python
  2. 逐行敲入下面的代码
    # Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
    如果能看到下面的输出
    Hello, TensorFlow!
    恭喜你，安装成功了……

更多的细节

请参考详细的官方文档

  1. Installing TensorFlow on Ubuntu
  2. Installing TensorFlow from Sources
  3. NVIDIA CUDA Installation Guide for Linux
  4. NVIDIA cuDNN
  5. CUDA GPUs
migege posted @ 2017-11-22 12:16:07 3187 views 0 comments TensorFlow Ubuntu GPU CUDA NVIDIA
©2016-2019 migege.com. 京ICP备16008672号-1
Proudly driven by MiLog & go1.17.3 - 10.640069ms