Tech on ottercoconut's Blog

Synology NAS Proxy and Emby Configuration

Mon, 16 Feb 2026 11:12:15 +0800

Recently, when using v2rayA on my Synology NAS, I encountered an issue where incorrect time synchronization caused it to stop working properly, which in turn prevented Emby from scraping metadata. Although I don’t know the exact reason, it’s an old problem. The version was very old, and I was using a Docker image from a third-party application store; currently, Docker can’t pull images via direct connection either. Here is my experience with solving this:

Reinstall v2rayA
Configure the Docker version of Emby.

Reference Websites:

Linux Backup Installation Method - v2rayA

XTLS/Xray-core: Xray, Penetrates Everything. Also the best v2ray-core. Where the magic happens. An open platform for various uses.

Implementing Transparent Proxy on Synology - v2rayA

sjtuross/syno-iptables: Some missing iptables modules for Synology

Setting up Proxy for Container Manager (Docker) on Synology DSM 7.2 - CSDN Blog

v2rayA

According to the official v2rayA documentation, there is no specially adapted version for Synology, so we use the generic binary file for installation. Since we cannot pull files from GitHub over a direct connection, we download it on our computer and transfer it to the NAS. In addition, the installation steps below are actually slightly different from the official documentation.

Copy v2rayA and xray

1
2
3
4


cp v2raya_linux_x64 /usr/local/bin/v2raya
cp xray /usr/local/bin/
chmod +x /usr/local/bin/v2raya
chmod +x /usr/local/bin/xray

Copy the GFWList proxy rule files included with xray: geoip.dat and geosite.dat
1 2 3

cd /usr/local/share mkdir xray cp .../g* xray

Modify the service configuration file

1

sudo vi /etc/systemd/system/v2raya.service

The content was generated by ChatGPT combined with the official documentation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


[Unit]
Description=v2rayA Service
After=network.target

[Service]
Type=simple
User=root
Environment="V2RAYA_CONFIG=/usr/local/etc/v2raya"
Environment="V2RAYA_LOG_FILE=/tmp/v2raya.log"
Environment="XRAY_LOCATION_ASSET=/usr/local/share/xray"
ExecStart=/usr/local/bin/v2raya --passcheckroot
Restart=on-failure
LimitNOFILE=1000000

[Install]
WantedBy=multi-user.target

Activate v2rayA. According to the official documentation, run it as a service
1 2

sudo systemctl start v2raya sudo systemctl status v2raya
Configure it to start up automatically on boot. Theoretically, the configuration is complete after this step. Along the way, I also solved the issue of transparency proxy not working on Synology.
1 2

sudo systemctl enable v2raya sudo systemctl is-enabled v2raya

Activate Transparent Proxy

arch	kernel	iptables version	system model	platform version
apollolake	4.4.180+	v1.8.3	DS918+	7.0.1-42218
apollolake	4.4.59+	v1.6.0	DS918+	6.2.3-25426
broadwell	3.10.105	v1.6.0	DS3617xs	6.2.3-25426
bromolow	3.10.105	v1.6.0	DS3615xs	6.2.3-25426
geminilake	4.4.180+	v1.8.3	DS920+	7.1-42661
geminilake	4.4.302+	v1.8.3	DS220+	7.2-64570

Due to the characteristics of the Synology system, first determine the architecture based on your model, then select the appropriate files for your machine from the downloaded modules from the GitHub repository (for example, my DS224 is geminilake).

Upload the corresponding ko modules to /lib/modules/, and upload the corresponding so modules to /usr/lib/iptables/, and that’s it.

Run sudo -i and then run the following insmod commands to attempt loading the ko kernel modules. Because the modules depend on each other, they must be loaded in a specific order.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


insmod /lib/modules/nfnetlink.ko
insmod /lib/modules/ip_set.ko
insmod /lib/modules/ip_set_hash_ip.ko
insmod /lib/modules/xt_set.ko
insmod /lib/modules/ip_set_hash_net.ko
insmod /lib/modules/xt_mark.ko
insmod /lib/modules/xt_connmark.ko
insmod /lib/modules/xt_comment.ko
insmod /lib/modules/xt_TPROXY.ko
insmod /lib/modules/xt_socket.ko
insmod /lib/modules/iptable_mangle.ko
insmod /lib/modules/textsearch.ko
insmod /lib/modules/ts_bm.ko
insmod /lib/modules/xt_string.ko

insmod /lib/modules/nf_nat_ipv6.ko
insmod /lib/modules/nf_nat_masquerade_ipv6.ko
insmod /lib/modules/ip6t_MASQUERADE.ko
insmod /lib/modules/ip6table_nat.ko
insmod /lib/modules/ip6table_raw.ko
insmod /lib/modules/ip6table_mangle.ko

However, after a system reboot, the modules need to be reloaded, so generate a script with the above content at /usr/local/bin/load_v2raya_mods.sh.

Then restart the v2rayA service:

1

sudo systemctl restart v2raya

Emby

After configuring v2rayA, even though I used the method of modifying the configuration file, Emby’s traffic still couldn’t successfully pass through v2rayA. After various fruitless attempts to modify the IP configuration, it naturally occurred to me that this was caused by Docker running as a host service.

After modifying the configuration on v2rayA multiple times, I finally found that enabling the transparent proxy solved the problem, and it also fixed the issue of being unable to connect to the Docker repository. But a bigger problem arose: enabling the transparent proxy caused all external network access to fail (fortunately, local network connections still worked). I think “transparent proxy” can be understood as a “global proxy”.

For the issue of setting up a proxy specifically for Docker (Container Manager), please refer to the links in the preface.

Select Emby in “Images” and run it. Add the path to the media library in “Volume Settings”, add three environment variables in “Environment”, and select host mode in “Network”:
1 2 3

HTTP_PROXY = http://127.0.0.1:20171 HTTPS_PROXY = http://127.0.0.1:20171 NO_PROXY = localhost,127.0.0.1
Complete the basic setup, install the previous anime-related plugins, and the scraping was successful.

How to Use Host Proxy on WSL2

Fri, 30 Jan 2026 16:31:20 +0800

Since I’ve been frequently downloading various models with Python recently, I finally decided to solve the long-standing issue of WSL2 not being able to use the Windows host’s proxy. After wasting half an afternoon, I finally got it working. With the help of my capable assistant Gemini, the main steps are as follows:

.wslconfig settings in Windows
Core settings of v2rayN
Firewall settings in Windows
Cleaning up old settings in WSL2
~/.bashrc settings in WSL2
curl -v testing in WSL2

References:

Advanced settings configuration in WSL | Microsoft Learn

WSL2 使用 V2RayN 局域网 proxychains 代理方案 · Issue #2653 · 2dust/v2rayN

记一次用wsl2中共享宿主机的代理-v2rayN - 沉迷于学习，无法自拔^_^

.wslconfig

Access your personal account folder in Windows. Press Win + R, enter %UserProfile%, and hit Enter.
Check if there is a .wslconfig file. If not, create a new text file and name it .wslconfig.

Paste the following content into .wslconfig:

1
2
3
4
5
6
7


[wsl2]
# Enable mirrored networking mode
networkingMode=mirrored
# Allow WSL2 to access Windows localhost
localhostForwarding=true
# Automatically synchronize proxy settings (optional: true/false)
autoProxy=true

Note that the autoProxy parameter determines how WSL2 handles the proxy. Setting it to true means you don’t need to configure ~/.bashrc anymore, but the problems are:

When we use the env | grep -i proxy command, we will see many strange network-related variables, even though the proxy does successfully work.
The proxy cannot be toggled on or off within WSL2, causing a lot of traffic to go through the proxy unnecessarily.

Later, we will set it to false so that we can easily control the proxy switch inside WSL2, ensuring a clear and transparent WSL2 system.

In the Windows terminal, enter wsl --shutdown to shut down WSL2.

v2rayN

In the basic settings of v2rayN (version V7.15.7 at the time of writing), enable “Allow connections from the LAN” and “Open a new port for the LAN” (optional).
At the bottom left of the v2rayN client’s main interface, you can see the port open for the internet, which is 10810 in my case.
The system proxy here is “Set system proxy automatically”, and the routing mode is “Bypass (Whitelist)”.
Then select a node and keep v2rayN running.

FireWall

Type “firewall” in the Windows search box and select Windows Defender Firewall.
Click “Allow an app or feature through Windows Defender Firewall”.
Find or add v2rayN.exe and its core programs (such as v2ray.exe or xray.exe), ensuring that both Private and Public are checked. The usual paths are under v2rayN\ and v2rayN\bin\xray, respectively.
In the advanced settings of the firewall, create a new inbound rule to allow TCP traffic on port 10810 (corresponding to v2rayN).

wsl2

Cleaning Up Old Settings

1
2
3
4
5
6
7
8


# 1. Clear old, messy proxy environment variables
unset http_proxy
unset https_proxy
unset no_proxy

# 2. Re-set correct proxy (pointing to localhost port in mirrored mode)
export http_proxy="http://127.0.0.1:10810"
export https_proxy="http://127.0.0.1:10810"

Ensure that you have enabled mirrored networking mode in the first step and successfully restarted WSL2. Of course, this step is not strictly necessary either, because ~/.bashrc will automatically handle these old variables later.

~/.bashrc

1
2


# Same applies to other editors
sudo vim ~/.bashrc

After multiple experiments with Gemini, here is the content to append to the bottom of ~/.bashrc:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


function proxy_on() {
 # Thoroughly clean up any residual variables (prevent conflicts from mixed case)
 unset http_proxy https_proxy ALL_PROXY NO_PROXY HTTP_PROXY HTTPS_PROXY all_proxy no_proxy
 
 # Set the port you verified works (since 10810 was tested successfully, use 10810)
 export hostip="127.0.0.1"
 export port="10810"
 
 export http_proxy="http://$hostip:$port"
 export https_proxy="http://$hostip:$port"
 export all_proxy="socks5://$hostip:$port"
 
 # no_proxy here only keeps localhost
 export no_proxy="localhost,127.0.0.1"
 
 echo "WSL Proxy: ON (127.0.0.1:10810)"
}

function proxy_off() {
 unset http_proxy https_proxy ALL_PROXY NO_PROXY HTTP_PROXY HTTPS_PROXY all_proxy no_proxy
 echo "WSL Proxy: OFF"
}

After saving the above content, enter:

1

source ~/.bashrc

And then:

1

curl -I https://www.google.com

The following output should appear, indicating a successful connection:

I also tested it with ping, which didn’t work but doesn’t affect its usability.

1

HTTP/1.1 200 Connection established

I tried downloading the model again, and it successfully worked:

1
2


import gensim.downloader as api
wv_from_bin = api.load("glove-wiki-gigaword-200")

Solution If It Still Fails

The most effective method is, after completing the above steps, to enter the following in WSL2:

1

curl -v https://www.google.com

Then feed the output result to an AI, and it will tell you what to do.

PyTorch Basics

Thu, 22 Jan 2026 10:49:28 +0800

This post covers the learning of basic PyTorch usage, primarily based on Bilibili Xiaotudui’s https://www.bilibili.com/video/BV1hE411t7RN/ and with the help of Gemini. In addition to practical applications, it also includes the principles of some deep learning models.

Dataset

An abstract class representing a Dataset.

All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite __getitem__, supporting fetching a data sample for a given key. Subclasses could also optionally overwrite __len__, which is expected to return the size of the dataset by many ~torch.utils.data.Sampler implementations and the default options of ~torch.utils.data.DataLoader. Subclasses could also optionally implement __getitems__, for speedup batched samples loading. This method accepts list of indices of samples of batch and returns list of samples.

note

~torch.utils.data.DataLoader by default constructs an index sampler that yields integral indices. To make it work with a map-style dataset with non-integral indices/keys, a custom sampler must be provided.

An abstract class representing a dataset.

All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite __getitem__, supporting fetching a data sample for a given key. Subclasses could also optionally overwrite __len__, which is expected to return the size of the dataset by many ~torch.utils.data.Sampler implementations and the default options of ~torch.utils.data.DataLoader. Subclasses could also optionally implement __getitems__, for speedup batched samples loading. This method accepts list of indices of samples of batch and returns list of samples.

Note

By default, ~torch.utils.data.DataLoader constructs an index sampler that yields integral indices. To make it work with a map-style dataset with non-integral indices/keys, a custom sampler must be provided.

Custom Dataset

So in actual usage, suppose there is a dataset (let’s use the previously used fer2013 as an example):

First, define a data class inheriting from Dataset, and define __init__()

1
2
3
4
5
6
7
8


from torch.utils.data import Dataset
import os
class MyData(Dataset):
 def __init__(self, root_dir, label_dir):
 self.root_dir = root_dir # E.g., fer2013/test
 self.label_dir = label_dir # E.g., angry
 self.path = os.path.join(self.root_dir, self.label_dir) # Path merging function, solves cross-system file path issues
 self.img_path = os.listdir(self.path) # E.g., the string list of all image names under the angry folder

Override __getitem()

1
2
3
4
5
6


def __getitem__(self, index):
 img_name = self.img_path[index]
 img_item_path = os.path.join(self.root_dir, self.label_dir, img_name) # Concatenate the specific path for a certain image
 img = Image.open(img_item_path)
 label = self.label_dir
 return img, label # Return an object (image) from the dataset and its type

Override __len()__

1
2


def __len__(self):
 return len(self.img_path)

Definition Example

1
2
3


root_dir = "fer2013/train"
angry_label_dir = "angry"
angry_dataset = MyData(root_dir, angry_label_dir)

Merging Datasets

1
2
3
4


disguest_label_dir = "disguest"
disgust_dataset = MyData(root_dir, disguest_label_dir)

train_dataset = angry_dataset + disgust_dataset # '+'

TensorBoard

SummaryWriter

Create a log directory
Initialize the Writer object
Generate event files
- This file is the real database (events.out……). When writer.add_scalar is subsequently called, the data is not drawn directly on the screen, but appended to this file.
- Once you finish executing the corresponding code and run tensorboard --logdir=logs, TensorBoard’s backend server reads the contents of this file and renders it into charts in the browser.

1
2
3


from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter("logs") # This line of code will create a folder named "logs" in your project's root directory

Note that you can use a different folder to record data for each experiment. For instance, writer = SummaryWriter("logs/lr0.01_batch32"), and after modifying the learning rate, writer = SummaryWriter("logs/lr0.001_batch32").

add_scalar()

Drawing function add_scalar()

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


(method) def add_scalar(
 tag: Any,
 scalar_value: Any, # Corresponds to the y-axis of the image
 global_step: Any | None = None, # Corresponds to the x-axis of the image
 walltime: Any | None = None,
 new_style: bool = False,
 double_precision: bool = False
) -> None

​``` Example ```

for i in range(100):
 writer.add_scalar("y=2x", 2*i, i)
writer.close()

add_image()

Image viewing function add_image()

1
2
3
4
5
6
7


(method) def add_image(
 tag: Any,
 img_tensor: Any,
 global_step: Any | None = None,
 walltime: Any | None = None,
 dataformats: str = "CHW"
) -> None

Pay attention to the type requirements for the parameters: img_tensor (torch.Tensor, numpy.ndarray, or string/blobname): Image data, so images of PIL type need to be converted, for instance using numpy. Also mind the data shape requirements: Tensor with :math:(1, H, W), :math:(H, W), :math:(H, W, 3) is also suitable as long as corresponding dataformats argument is passed, e.g. CHW, HWC, HW. Meaning three image data formats where the order of number of channels, height, and width differ.

1
2
3
4
5


img = Image.open(image_path)
img_array = np.array(img)
print(f"Image shape: {img_array.shape}")
writer.add_image("test",img_array,1,dataformats='HW') # Obvious from the .shape
writer.close()

add_graph()

Common Transforms

The following are mostly image processing methods.

ToTensor

Convert a PIL Image or ndarray to tensor and scale the values accordingly.

This transform does not support torchscript.

Converts a PIL Image or numpy.ndarray (H x W x C) in the range [0, 255] to a torch.FloatTensor of shape (C x H x W) in the range [0.0, 1.0] if the PIL Image belongs to one of the modes (L, LA, P, I, F, RGB, YCbCr, RGBA, CMYK, 1) or if the numpy.ndarray has dtype = np.uint8

In the other cases, tensors are returned without scaling.

note

Because the input image is scaled to [0.0, 1.0], this transformation should not be used when transforming target image masks. See the [references](vscode-file://vscode-app/d:/Microsoft VS Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) for implementing the transforms for image masks.
Convert PIL images to torch type, such as torch.Size([1, 48, 48])

1
2
3
4
5


trans_totensor = transforms.ToTensor()
img_tensor = trans_totensor(img)
print(img_tensor.shape)
writer.add_image("ToTensor",img_tensor)
writer.close()

Normalize

Normalize a tensor image with mean and standard deviation.

This transform does not support PIL Image.

Given mean: (mean[1],...,mean[n]) and std: (std[1],..,std[n]) for n

channels, this transform will normalize each channel of the input

torch.*Tensor i.e.,

output[channel] = (input[channel] - mean[channel]) / std[channel]

note:

This transform acts out of place, i.e., it does not mutate the input tensor.

Args:

mean (sequence): Sequence of means for each channel.

std (sequence): Sequence of standard deviations for each channel.

inplace(bool,optional): Bool to make this operation in-place.

The mathematical principle of Normalize is:

$$ output = \frac{input-mean}{std} $$

The parameter mean affects the “center position”. After ToTensor, pixels range from $[0, 1]$, centered around $0.5$. If mean = 0.5, subtracting $0.5$ shifts the data center to $0$. The original range of $[0, 1]$ becomes $[-0.5, 0.5]$.
The parameter std affects the “scaling magnitude”. E.g., std = 0.5 means dividing the data range by $0.5$, making the final range $[-1, 1]$ from $[-0.5, 0.5]$.

Resize

Resize the input image to the given size.

If the image is torch Tensor, it is expected

to have […, H, W] shape, where … means a maximum of two leading dimensions

Args:

size (sequence or int): Desired output size. If size is a sequence like

(h, w), output size will be matched to this. If size is an int,

smaller edge of the image will be matched to this number.

i.e, if height > width, then image will be rescaled to

(size * height / width, size).

resize() supports both PIL and Tensor image formats. If it’s a Tensor, the expected shape is [..., H, W]. The ... here indicates it can handle [C, H, W] (single image) or [B, C, H, W] (a Batch of images).
The parameter size needs to be written as a sequence, like resize((512, 512)). If only one parameter is input, like resize(512), the image’s shorter edge becomes 512, while the longer edge scales proportionally.

1
2
3
4


print(img.size)
trans_resize = transforms.Resize((224,224))
img_resize = trans_resize(img)
print(img_resize)

Compose

Composes several transforms together. This transform does not support torchscript.

Please, see the note below.

Args:

transforms (list of Transform objects): list of transforms to compose.

The Compose() operation is a pipeline class for various transforms operations.
In deep learning, images usually go through a fixed series of steps (e.g., resizing -> converting to Tensor -> normalizing). If Compose isn’t used, you have to manually call multiple functions for every image, making the code highly redundant.
The argument type for Compose() is a list, and the operations fire sequentially, so the data type output by the previous operation must be acceptable as input for the next.

1
2
3
4
5
6
7
8
9


from torchvision import transforms
# Define preprocessing for the training set
train_transform = transforms.Compose([
 transforms.Resize((224, 224)), # VGG16 standard input is 224x224
 transforms.RandomHorizontalFlip(), # Data augmentation: Random horizontal flip
 transforms.ToTensor(), # Normalize to [0.0, 1.0]
 transforms.Normalize([0.5], [0.5]) # Standardize to [-1.0, 1.0]
])
img_tensor = train_transform(img)

PyTorch Dataset Usage

For instance, to import a dataset for computer vision learning, we can directly download the dataset within the program.

1
2
3
4
5


import torchvision

train_set = torchvision.datasets.CIFAR10(root="./dataset...", train=True, download=True)

test_set = torchvision.datasets.CIFAR10(root="./dataset...", train=false, download=True)

The root parameter indicates where the dataset is stored, train specifies whether the dataset is for training, and download indicates whether to download it locally (it generates a download link).
Specific parameter configurations may differ for each dataset…
If the dataset has already been downloaded locally, it can be copied into the project’s dataset directory, saving download time upon running.

1
2
3
4
5
6


print(test_set.classes) # You can see all the categories in the test dataset

img, target = test_set[0]
print(img)
print(target)
print(test_set.classes[target]) # Output the category corresponding to the first element in the test set

DataLoader

Data loader combines a dataset and a sampler, and provides an iterable over the given dataset.

The :class:~torch.utils.data.DataLoader supports both map-style and

iterable-style datasets with single- or multi-process loading, customizing

loading order and optional automatic batching (collation) and memory pinning.

When training a model, a massive volume of data from the dataset cannot be crammed into memory all at once. DataLoader achieves:

Batching: Packages images into groups (Batches).
Shuffling: Shuffles data randomly at the start of every training epoch, ensuring the model prevents rote memorization of the data’s ordering.
Parallel Computing: Leverages multi-core CPUs to pre-prepare proceeding batches of data, letting GPUs avoid idle time waiting.

Parameter	Common Values	Description
dataset	Custom Dataset	Required. Tells DataLoader which “warehouse” to fetch data from.
batch_size	16, 32, 64…	Number of samples loaded per batch. The larger it is, the faster the training, but it consumes more VRAM. In FER emotion recognition, 32 or 64 are common values.
shuffle	`True` / `False`	Whether to shuffle the order. Training sets are usually set to `True` (adding randomness); test sets are usually set to `False`.
num_workers	0, 2, 4, 8…	Multi-process loading. `0` means only the main process is used (slow). Increasing the value speeds up read times. Recommendation: Set to half of your CPU cores.
drop_last	`True` / `False`	Drop the last incomplete batch. E.g., if there are 100 images and `batch_size=32`. The remaining 4 images aren’t enough for a batch. Setting this to `True` discards these 4, ensuring each Batch size is consistent.
pin_memory	`True`	Page-locked memory. If training on a GPU, setting to `True` accelerates data transfer speeds from RAM to VRAM.

For example, we use DataLoader to process data from CIFAR10.

1
2
3


test_data = torchvision.datasets.CIFAR10("./dataset", train=False, transform=torchvision.transforms.ToTensor(), download=True)

test_loader = DataLoader(dataset=test_data, batch_size=4, shuffle=True, num_workers=0, drop_last=False)

Combining loops and tensorboard, output images used in each step arrayed across every epoch.
In the code below, step + epoch * len(test_loader) utilizes a global step size, but alternative setups skipping this to treat each epoch identically completely as distinct grouped iterations behave comparably similarly.

1
2
3
4
5
6
7
8


writer = SummaryWriter("dataloader")
for epoch in range(2):
 step = 0
 for data in test_loader:
 imgs, targets = data
 writer.add_images("test_data_batch", imgs, step + epoch * len(test_loader))
 step = step + 1
writer.close()

nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure.

In PyTorch, whether it’s a simple linear layer or a complex VGG16 or Transformer, they are all essentially an nn.Module. It is the base class for all neural network modules.
nn.Module supports nesting. When you call model.to("cuda") on a large model, PyTorch automatically traverses this “tree” and moves all its sub-layers to the GPU.
As long as you assign a layer to self.xxx in __init__, PyTorch will automatically identify its Weights and Bias and add them to the list of parameters to be optimized.

When writing a subclass of nn.Module, you must override __init__() and forward().

__init(self)__
- Define network layers here (convolution, pooling, fully connected, etc.).
- You must call super().__init__(). This line initializes the parent class’s properties; without it, PyTorch won’t be able to automatically track the defined layers, and the model won’t train.
forward(self, x)
- Defines the flow of data. Specifies which layers an image passes through sequentially.
- You do not need to manually call forward. Simply run model(input), and PyTorch will automatically trigger forward.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


import torch
from torch import nn

class myModule(nn.Module):
 def __init__(self):
 super().__init__()

 def forward(self, input):
 output = input + 1 # Simply add 1 to the input and output it
 return output

Convolution Conv

$$ \text{out}(N_i, C_{\text{out}_j}) = \text{bias}(C_{\text{out}_j}) + \sum_{k = 0}^{C_{\text{in}} - 1} \text{weight}(C_{\text{out}_j}, k) \star \text{input}(N_i, k) $$

Convolution animation page: conv_arithmetic/README.md at master · vdumoulin/conv_arithmetic

Parameter	Meaning	Function
in_channels	Input Channels	Usually 3 (RGB) for color images, and 1 for grayscale images.
out_channels	Output Channels	Number of convolution kernels. The number of kernels determines the number of layers in the output feature map.
kernel_size	Kernel Size	The size of the “window” used to extract features. Commonly 3 or 5 (VGG usually defaults to 3).
stride	Stride	The span at which the window slides. Defaults to 1. The larger the stride, the smaller the output image.
padding	Padding	Pads with 0s around the image. `'same'` keeps the size unchanged, while `'valid'` applies no padding.
dilation	Dilated Convolution	The spacing between points in the convolution kernel. Used to increase the receptive field (without increasing the number of parameters).
bias	Bias	Whether to add a constant offset to the result. Enabled by default.

The Padding parameter of convolution is very important. If zeros are not padded around it, the convolution will cause the image size to become smaller and smaller.
Shape calculation formula:

$$ H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor $$

$W_{out}$ calculations follow a similar approach.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


dataset = torchvision.datasets.CIFAR10("./dataset",train = False, transform=torchvision.transforms.ToTensor(),download=True)
dataloader = DataLoader(dataset, batch_size=64)

class myModule(nn.Module):
 def __init__(self):
 super().__init__()
 self.conv1 = Conv2d(in_channels=3,out_channels=6,kernel_size=3,stride=1,padding=0)

 def forward(self,x):
 x = self.conv1(x)
 return x

mymodule = myModule() # Model instantiation 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


step = 0
for data in dataloader:
 imgs, targets = data
 output = mymodule(imgs)
 print(imgs.shape)
 # torch.Size([64, 3, 32, 32])
 print(output.shape)
 # torch.Size([64, 6, 30, 30]) channel == 6 After convolution, the number of channels changes, so the image cannot be directly outputted

 step = step + 1

(Max) Pooling MaxPool

The logic of max pooling is extremely simple: within a window range (Kernel), only the largest value is kept, and the rest are discarded.
It preserves input features while simultaneously reducing data volume, speeding up training.

Parameter	Unique Features
kernel_size	Window size. Typically `2` (meaning merging a $2 \times 2$ region).
stride	Default value equals kernel_size! This differs from convolution. If `kernel_size=2`, the stride defaults to `2`, so the windows do not overlap.
ceil_mode	Very important. The default is `False` (floor). If set to `True` (ceiling), when the window exceeds boundaries, as long as there is data in the window, the result will be retained instead of discarded.
padding	Padding. Note that pooling pads with negative infinity ($-\infty$), ensuring padded spots are not selected as the maximum value.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


input = torch.tensor([[1,2,0,3,1],
 [0,1,2,3,1],
 [1,2,1,0,0],
 [5,2,3,1,1],
 [2,1,0,1,1]], dtype=float)

input = torch.reshape(input,(-1,1,5,5))
print(input.shape)

class myMoudle(nn.Module):
 def __init__(self, *args, **kwargs):
 super(myMoudle, self).__init__()
 self.maxpool1 = MaxPool2d(kernel_size=3, ceil_mode=True) 
 '''
 ceil_mode = False means it only takes the pooling result when the pooling kernel encounters the maximum expected size (e.g., 3x3), otherwise it discards it
 '''
 def forward(self, input):
 output = self.maxpool1(input)
 return output
 
mymoudle = myMoudle()
output = mymoudle(input)
print(output)

1
2
3


torch.Size([1, 1, 5, 5])
tensor([[[[2., 3.],
 [5., 1.]]]], dtype=torch.float64)

Loss Functions and Backpropagation

The loss function is used to calculate the gap between the actual output and the target, providing a basis for backpropagation and parameter updates. In classification tasks, the cross-entropy function is commonly used to calculate the error.

1

loss_func = nn.CrossEntropyLoss()

Optimizer

torch.optim — PyTorch 2.10 documentation

Example code:

1
2
3
4
5
6


for input, target in dataset:
 optimizer.zero_grad() # Clear gradients
 output = model(input)
 loss = loss_fn(output, target) # Call loss function
 loss.backward() # Backpropagation
 optimizer.step() 

PyTorch in Practice: CIFAR10

A practical example of a simple classification model targeting the CIFAR10 image dataset.

First, let’s understand Sequential. nn.Sequential is a special subclass of nn.Module whose purpose is to automatically execute the forward logic. Note: each parameter inside is a class for a certain layer, meaning they must be comma-separated. Sequential simplifies both the model definition and the forward() operation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


class CIFAR10_Simple(nn.Module):
 def __init__(self, *args, **kwargs):
 super(CIFAR10_Simple, self).__init__(*args, **kwargs)
 self.conv1 = Conv2d(in_channels=3, out_channels=32, kernel_size=5, padding=2)
 '''
 The value of the padding parameter can be derived from visualizing the image: for a 5x5 convolution kernel, when centered at the image's (0,0), the kernel extends outwards by 2 units. This is a simplified estimation method; in reality, you should substitute values into the dimension formula for calculation (refer to the "Convolution Conv" section).
 '''
 self.model_s = Sequential(
 Conv2d(in_channels=3, out_channels=32, kernel_size=5, padding=2),
 nn.ReLU(),
 MaxPool2d(kernel_size=2),
 Conv2d(in_channels=32, out_channels=32, kernel_size=5, padding=2),
 nn.ReLU(),
 MaxPool2d(2),
 Conv2d(32,64,5,padding=2),
 nn.ReLU(),
 MaxPool2d(2),
 Flatten(),
 Linear(1024, 64),
 nn.ReLU(),
 Linear(64, 10)
 )

 def forward(self, x):
 '''
 x = self.conv1(x)
 x = self.maxpool1(x)
 x = self.conv2(x)
 x = self.maxpool2(x)
 x = self.conv3(x)
 x = self.maxpool3(x)
 x = self.flatten(x)
 x = self.linear1(x)
 x = self.linear2(x)
 '''
 x = self.model_s(x)
 return x

Before building the model, initially set up DataLoader to handle the dataset.
- Prepare train_data and test_data independently respectively.

1
2
3
4
5
6


dataset_transfrom = tf.Compose([tf.ToTensor(),tf.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))])
train_data = torchvision.datasets.CIFAR10("./dataset", transform=dataset_transfrom, download=True)
test_data = torchvision.datasets.CIFAR10("./dataset", train=False, transform=dataset_transfrom, download=True)
# --------
train_loader = DataLoader(dataset=train_data, batch_size=64, shuffle=True, drop_last=True)
test_loader = DataLoader(dataset=test_data, batch_size=64, shuffle=True, drop_last=True)

After building out the model properly, run a trivial check testing evaluating exactly asserting checking whether output dimension correctly matches expectation guidelines appropriately cleanly.

1
2
3
4
5


cifar = CIFAR10_Simple()
print(cifar)
input = torch.ones((64, 3, 32, 32)) # Test with identical dataset picture sizes
output = cifar(input)
print(output.shape)

Basic settings before training
- Define TensorBoard writer
- Setup device to call the graphics card for accelerated training
- Instantiate the training model
- Define the loss function
- Set up the optimizer

1
2
3
4
5
6


writer = SummaryWriter("./logs")
writer.add_graph(cifar, input)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = CIFAR10_Simple().to(device)
loss_func = nn.CrossEntropyLoss() # Cross-entropy loss function
optim = torch.optim.SGD(cifar.parameters(), lr=0.01) # Learning rate

Training section
- Boilerplate code for the optimizer
- Record training loss

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


total_step = 0
for epoch in range(10):
 # --- Training section ---
 model.train()
 for data in train_loader:
 imgs, targets = data
 outputs = model(imgs.to(device))
 loss = loss_func(outputs, targets.to(device))
 
 optim.zero_grad()
 loss.backward()
 optim.step()
 
 # Record training loss
 writer.add_scalar("Train_Loss", loss.item(), total_step)
 total_step += 1

Evaluation section
- Execute once per epoch
- with torch.no_grad(): Turn off gradient recording
- Compile performance metrics

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


model.eval()
 total_test_loss = 0
 total_accuracy = 0
 with torch.no_grad(): # No need to calculate gradients during testing, saves performance
 for data in test_loader:
 imgs, targets = data
 imgs, targets = imgs.to(device), targets.to(device)
 outputs = model(imgs)
 
 # Calculate total loss
 loss = loss_func(outputs, targets)
 total_test_loss += loss.item()
 
 # Calculate accuracy: argmax(1) finds the category index with the highest probability
 accuracy = (outputs.argmax(1) == targets).sum()
 total_accuracy += accuracy

Visualization

1
2
3
4
5
6
7


# Output to TensorBoard
 writer.add_scalar("Test_Loss", total_test_loss / len(test_loader), epoch)
 writer.add_scalar("Test_Accuracy", total_accuracy / len(test_data), epoch)
 
 print(f"Epoch {epoch+1} finished, Accuracy: {total_accuracy / len(test_data)}")

writer.close()

Maven Fundamentals

Tue, 01 Apr 2025 14:49:00 +0800

Reference Websites

Maven Fundamentals - Java Tutorial - Liao Xuefeng’s Official Website

Maven Fundamentals

Introduction to Maven

Before understanding Maven, let’s look at what a Java project needs. First, we need to determine which dependency packages to introduce. For example, if we need to use commons logging, we must put the commons logging jar package into the classpath. If we also need log4j, we need to put all log4j-related jar packages into the classpath. This is dependency management.

Secondly, we must determine the directory structure of the project. For example, the src directory stores Java source code, the resources directory stores configuration files, and the bin directory stores the compiled .class files.

Furthermore, we also need to configure the environment, such as the JDK version, the compilation and packaging process, and the version number of the current code.

Finally, in addition to using IDEs like Eclipse for compilation, we must also be able to compile via command-line tools so that the project can be compiled, tested, and deployed on an independent server.

These tasks are not difficult, but they are very tedious and time-consuming. If every project had its own set of configurations, it would certainly be a mess. What we need is a standardized Java project management and build tool.

Maven is a management and build tool specifically created for Java projects. Its main features include:

Providing a standardized project structure;
Providing a standardized build process (compilation, testing, packaging, publishing…);
Providing a dependency management mechanism.

Maven Project Structure

A typical Java project managed using Maven has the following default directory structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


a-maven-project
├── pom.xml
├── src
│ ├── main
│ │ ├── java
│ │ └── resources
│ └── test
│ ├── java
│ └── resources
└── target

The root directory of the project a-maven-project is the project name. It has a project description file pom.xml. The directory storing Java source code is src/main/java, the directory storing resource files is src/main/resources, the directory storing test source code is src/test/java, and the directory storing test resources is src/test/resources. Finally, all files generated by compilation and packaging are placed in the target directory. These constitute the standard directory structure of a Maven project.

All directory structures are agreed-upon standard structures, and we must never modify the directory structure arbitrarily. Using the standard structure requires no configuration, and Maven can be utilized normally.

Let’s look at the most critical project description file, pom.xml. Its content looks like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


<project ...>
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.itranswarp.learnjava</groupId>
	<artifactId>hello</artifactId>
	<version>1.0</version>
	<packaging>jar</packaging>
	<properties>
 <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<maven.compiler.release>17</maven.compiler.release>
	</properties>
	<dependencies>
 <dependency>
 <groupId>org.slf4j</groupId>
 <artifactId>slf4j-simple</artifactId>
 <version>2.0.16</version>
 </dependency>
	</dependencies>
</project>

Here, groupId is similar to Java’s package name, typically the name of a company or organization. artifactId is similar to a Java class name, typically the project name. Together with version, a Maven project is uniquely identified by groupId, artifactId, and version.

When we reference other third-party libraries, it is also determined through these 3 variables. For example, depending on org.slfj4:slf4j-simple:2.0.16:

1
2
3
4
5


<dependency>
 <groupId>org.slf4j</groupId>
 <artifactId>slf4j-simple</artifactId>
 <version>2.0.16</version>
</dependency>

After declaring a dependency using <dependency>, Maven will automatically download this dependency package and put it into the classpath.

Additionally, note that <properties> defines some properties. Commonly used properties are:

project.build.sourceEncoding: Indicates the character encoding of the project source code, typically set to UTF-8;
maven.compiler.release: Indicates the JDK version to use, for example, 21;
maven.compiler.source: Indicates the source code version read by the Java compiler;
maven.compiler.target: Indicates the Class version compiled by the Java compiler.

Starting from Java 9, it is recommended to use the maven.compiler.release property to ensure that the input source code and the compiled output version are consistent during compilation. If the source code and output versions are different, maven.compiler.source and maven.compiler.target should be set respectively.

By defining properties via <properties>, the JDK version can be fixed, preventing different developers of the same project from using different versions of the JDK.

Summary

Maven is a management and build tool for Java projects:

Maven uses pom.xml to define project content and utilizes a preset directory structure;
Declaring a dependency in Maven can automatically download and import it into the classpath;
Maven uses groupId, artifactId, and version to uniquely locate a dependency.

Dependency Management

If our project depends on third-party jar packages, such as commons logging, the question arises: where do we download the published jar package for commons logging?

If we also want to depend on log4j, what jar packages are needed to use log4j?

Similar dependencies include: JUnit, JavaMail, MySQL driver, etc. A feasible method is to search for the project’s official website via a search engine, manually download the zip package, extract it, and put it into the classpath. However, this process is very tedious.

Maven solves the dependency management problem. For example, our project depends on the jar package abc, and abc depends on the jar package xyz:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


┌──────────────┐
│Sample Project│
└──────────────┘
 │
 ▼
┌──────────────┐
│ abc │
└──────────────┘
 │
 ▼
┌──────────────┐
│ xyz │
└──────────────┘

When we declare the dependency for abc, Maven automatically adds both abc and xyz to our project dependencies. We don’t need to manually investigate whether abc requires xyz.

Therefore, the first role of Maven is to resolve dependency management. We declare that our project needs abc, Maven will automatically import the jar package of abc, then determine that abc needs xyz, and will automatically import the jar package of xyz. Thus, ultimately, our project will depend on both abc and xyz jar packages.

Let’s look at a complex dependency example:

1
2
3
4
5


<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-web</artifactId>
 <version>1.4.2.RELEASE</version>
</dependency>

When we declare a spring-boot-starter-web dependency, Maven will automatically parse and determine that approximately twenty or thirty other dependencies are ultimately required:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


spring-boot-starter-web
 spring-boot-starter
 spring-boot
 sprint-boot-autoconfigure
 spring-boot-starter-logging
 logback-classic
 logback-core
 slf4j-api
 jcl-over-slf4j
 slf4j-api
 jul-to-slf4j
 slf4j-api
 log4j-over-slf4j
 slf4j-api
 spring-core
 snakeyaml
 spring-boot-starter-tomcat
 tomcat-embed-core
 tomcat-embed-el
 tomcat-embed-websocket
 tomcat-embed-core
 jackson-databind
 ...

If we try to manually manage these dependencies ourselves, it is extremely time-consuming, laborious, and the probability of errors is very high.

Dependency Relationships

Maven defines several dependency relationships, namely compile, test, runtime, and provided:

scope	Description	Example
compile	This jar package is needed during compilation (default)	commons-logging
test	This jar package is needed when compiling tests	junit
runtime	Not needed during compilation, but required at runtime	mysql
provided	Needed during compilation, but provided by JDK or a server at runtime	servlet-api

Among them, the default compile is the most commonly used, and Maven will place dependencies of this type directly into the classpath.

test dependencies indicate they are only used during testing and are not needed during normal execution. The most common test dependency is JUnit:

1
2
3
4
5
6


<dependency>
 <groupId>org.junit.jupiter</groupId>
 <artifactId>junit-jupiter-api</artifactId>
 <version>5.3.2</version>
 <scope>test</scope>
</dependency>

runtime dependencies indicate they are not needed during compilation but are required at runtime. The most typical runtime dependencies are JDBC drivers, such as the MySQL driver:

1
2
3
4
5
6


<dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
 <version>5.1.48</version>
 <scope>runtime</scope>
</dependency>

provided dependencies indicate they are needed during compilation but not at runtime. The most typical provided dependency is the Servlet API, which is needed during compilation; however, at runtime, the Servlet server has built-in related jars, so they are not needed during the execution phase:

1
2
3
4
5
6


<dependency>
 <groupId>jakarta.servlet</groupId>
 <artifactId>jakarta.servlet-api</artifactId>
 <version>4.0.0</version>
 <scope>provided</scope>
</dependency>

The last question is, how does Maven know where to download the required dependencies? That is, the related jar packages? The answer is that Maven maintains a central repository (repo1.maven.org), where all third-party libraries upload their own jars and related information. Maven can download the required dependencies from the central repository to the local machine.

Maven does not download jar packages from the central repository every time. Once a jar package has been downloaded, it is automatically cached by Maven in a local directory (the .m2 directory in the user’s home directory). Therefore, apart from the first compilation being relatively slow due to the time needed for downloading, subsequent processes will not repeatedly download the same jar packages because of the local cache.

Unique ID

For any given dependency, Maven only needs 3 variables to uniquely identify a jar package:

groupId: The name of the organization it belongs to, similar to a Java package name;
artifactId: The name of the jar package itself, similar to a Java class name;
version: The version of the jar package.

Through the above 3 variables, a certain jar package can be uniquely determined. Maven ensures that any jar package cannot be modified once published by performing PGP signing on the jar packages. The only way to modify a published jar package is to publish a new version.

Therefore, once a jar package has been downloaded by Maven, it can be permanently and safely cached locally.

Note: Only version numbers ending with -SNAPSHOT are regarded by Maven as development versions. Development versions are repeatedly downloaded every time. Such SNAPSHOT versions can only be used in internal private Maven repos, and publicly published versions are not allowed to be SNAPSHOTs.

Henceforth, when we represent Maven dependencies, we use the abbreviated form groupId:artifactId:version, for example: org.slf4j:slf4j-api:2.0.4.

Maven Mirrors

Besides downloading from Maven’s central repository, you can also download from Maven mirror repositories. If accessing Maven’s central repository is very slow, we can choose a faster Maven mirror repository. Maven mirror repositories synchronize periodically from the central repository:

1
2
3
4
5
6
7
8
9


 slow ┌───────────────────┐
 ┌─────────────▶│Maven Central Repo.│
 │ └───────────────────┘
 │ │
 │ │sync
 │ ▼
┌───────┐ fast ┌───────────────────┐
│ User │─────────▶│Maven Mirror Repo. │
└───────┘ └───────────────────┘

Users in the China region can use the Maven mirror repository provided by Alibaba Cloud. Using a Maven mirror repository requires configuration. In the user’s home directory, enter the .m2 directory and create a settings.xml configuration file with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


<settings>
 <mirrors>
 <mirror>
 <id>aliyun</id>
 <name>aliyun</name>
 <mirrorOf>central</mirrorOf>
 <!-- Alibaba Cloud's Maven mirror is recommended in China -->
 <url>https://maven.aliyun.com/repository/central</url>
 </mirror>
 </mirrors>
</settings>

After configuring the mirror repository, Maven’s downloading speed will be very fast.

Searching for Third-Party Components

The final question: if we want to reference a third-party component, such as okhttp, how do we exactly acquire its groupId, artifactId, and version? The method is to search for keywords via search.maven.org. After finding the corresponding component, directly copy it.

Command Line Compilation

In the command line, navigate to the directory where pom.xml is located, and enter the following command:

1

$ mvn clean package

If everything goes smoothly, you can obtain the automatically packaged jar after compilation in the target directory.

Using Maven in an IDE

Almost all IDEs have built-in support for Maven. In Eclipse, you can directly create or import a Maven project. If the imported Maven project has errors, you can try selecting the project, right-clicking, and choosing Maven - Update Project… to update it.

Summary

Maven determines the jar packages required by the project through parsing dependency relationships. The 4 commonly used scopes are: compile (default), test, runtime, and provided;

Maven downloads the required jar packages from the central repository and caches them locally;

Downloading can be accelerated through mirror repositories.

Build Process

Maven not only has a standardized project structure, but it also has a standardized build process that can automatically automate compiling, packaging, publishing, and more.

Lifecycle and Phase

When using Maven, we first need to understand what Maven’s lifecycle is.

Maven’s lifecycle consists of a series of phases. Taking the built-in lifecycle default as an example, it includes the following phases:

validate
initialize
generate-sources
process-sources
generate-resources
process-resources
compile
process-classes
generate-test-sources
process-test-sources
generate-test-resources
process-test-resources
test-compile
process-test-classes
test
prepare-package
package
pre-integration-test
integration-test
post-integration-test
verify
install
deploy

If we run mvn package, Maven will execute the default lifecycle, and it will run consistently from the beginning up until the package phase:

validate
initialize
…
prepare-package
package

If we run mvn compile, Maven will also execute the default lifecycle, but this time it will only run up to compile, namely the following phases:

validate
initialize
…
process-resources
compile

Another commonly used lifecycle in Maven is clean, which executes 3 phases:

pre-clean
clean (note that this clean is a phase, not a lifecycle)
post-clean

Therefore, when we use the mvn command, the parameter following it is a phase, and Maven automatically runs up to the specified phase according to the lifecycle.

A more complex example is specifying multiple phases. For example, running mvn clean package, Maven first executes the clean lifecycle and runs up to the clean phase, then it executes the default lifecycle and runs up to the package phase. The actually executed phases are as follows:

pre-clean
clean (note that this clean is a phase)
validate (starts executing the first phase of the default lifecycle)
initialize
…
prepare-package
package

During the actual development process, frequently used commands include:

mvn clean: Cleans up all generated classes and jars;

mvn clean compile: Cleans first, then executes up to compile;

mvn clean test: Cleans first, then executes up to test. Because compile must be executed before executing test, there is no need to specify compile here;

mvn clean package: Cleans first, then executes up to package.

During the execution process of most phases, because we usually do not configure related settings in pom.xml, these phases effectively do nothing.

The phases that are frequently used are actually only a few:

clean: clean up
compile: compile
test: run tests
package: package

Goal

Executing a phase subsequently triggers one or multiple goals:

Executed Phase	Corresponding Executed Goal
compile	compiler:compile
test	compiler:testCompile surefire:test

The naming of a goal always takes the format of abc:xyz.

Actually, if we draw an analogy, it becomes clear:

lifecycle is equivalent to a Java package; it contains one or multiple phases;
phase is equivalent to a Java class; it contains one or multiple goals;
goal is equivalent to a class method; it is actually the one doing the real work.

In most cases, we simply specify the phase, and it defaults to executing the goals bound by default to these phases. Only in a few instances do we directly specify running a goal, for example, starting a Tomcat server:

1

$ mvn tomcat:run

Summary

Maven provides a standard build process through lifecycles, phases, and goals.

The most commonly used build command entails specifying a phase, subsequently allowing Maven to execute up to the designated phase:

mvn clean
mvn clean compile
mvn clean test
mvn clean package

Normally, we always execute the goals natively bound by default to the phase, so it is unnecessary to specify the goal.

Java Multithreading

Fri, 21 Mar 2025 14:50:07 +0800

Reference Websites

Multithreading - Java Tutorial - Liao Xuefeng’s Official Website

Mostly referenced Mr. Liao’s blog, a very good tutorial.

Detailed Illustration of Java Multithreading - Personal Article - SegmentFault

Relatively less detailed; covers synchronization locks and thread pools. Concise and clear.

Also supplemented some knowledge, such as thread status, synchronized locks, the producer-consumer model, etc.

Java Multithreading

Process/Thread

The relationship between processes and threads: A process can contain one or multiple threads, but there is always at least one thread.

The smallest execution unit scheduled by the operating system is actually a thread, not a process. Commonly used operating systems like Windows and Linux utilize preemptive multitasking. How a thread is scheduled is entirely determined by the OS; the program itself cannot decide when or for how long a thread executes.

Multitasking can be achieved by multi-processing, multi-threading within a single process, or a mix of multi-processing + multi-threading.

Compared to multi-threading, the disadvantages of multi-processing are:

Creating a process incurs more overhead than creating a thread, especially on Windows systems.
Inter-process communication is slower than inter-thread communication, as inter-thread communication merely involves reading and writing the same variables, which is extremely fast.

The advantages of multi-processing are:

Multi-processing has higher stability than multi-threading. In a multi-process scenario, the crash of one process does not affect others.
In a multi-threading scenario, the crash of any single thread directly causes the crash of the entire process.

Multithreading

The Java language has built-in support for multithreading: a Java program is actually a JVM process. The JVM process uses a main thread to execute the main() method, and within the main() method, we can start multiple threads. Furthermore, the JVM has other worker threads responsible for garbage collection, etc.

Compared to single-threaded programming, the characteristic of multithreaded programming is: multithreading often requires reading and writing shared data, which requires synchronization.

For instance, when playing a movie, one thread must play the video and another the audio. The two threads must coordinate their execution, otherwise, the imagery and audio will be out of sync. Therefore, multithreaded programming is highly complex and more difficult to debug.

Creating Multithreading

Creating a New Thread - Java Tutorial - Liao Xuefeng’s Official Website

Creating a new thread is very easy; we need to instantiate a Thread object, and then call its start() method:

1
2
3
4
5
6


public class Main {
 public static void main(String args[]) {
 Thread t = new Thread();
 t.start();
 }
}

To have the new thread execute specified code, there are several methods:

Method 1: Derive a custom class from Thread and then override the run() method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


public class Main {
 public static void main(String args[]) {
 Thread t = new Thread();
 t.start();
 }
}
class MyThread extends Thread {
 @Override
 public void run(){
 	System.out.println("start new thread!");
 }
}

When executing the above code, notice that the start() method will automatically invoke the instance’s run() method internally.

Method 2: When creating a Thread instance, pass in a Runnable instance.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


public class Main {
 public static void main(String[] args) {
 Thread t = new Thread(new MyRunnable());
 t.start(); // Start new thread
 }
}
class MyRunnable implements Runnable {
 @Override
 public void run() {
 System.out.println("start new thread!");
 }
}

Or further simplify it using the lambda syntax introduced in Java 8:

1
2
3
4
5
6
7
8


public class Main {
 public static void main(String[] args) {
 Thread t = new Thread(() -> {
 System.out.println("start new thread!");
 });
 t.start(); // Start new thread
 }
}

However, directly calling the run() method does not achieve multithreading, the current thread will not change; it merely executes the run() method.

You must call the start() method of the Thread instance to start a new thread. If we look at the source code of the Thread class, we see that the start() method internally invokes a private native void start0() method. The native modifier indicates that this method is implemented by C code inside the JVM virtual machine, not by Java code.

The difference between using a thread and executing directly in the main() method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


public class Main {
 public static void main(String[] args) {
 System.out.println("main start...");
 Thread t = new Thread() {
 public void run() {
 System.out.println("thread run...");
 System.out.println("thread end.");
 }
 };
 t.start();
 System.out.println("main end...");
 }
}

Execution order in main:

Print main start...
Create Thread object
start invokes the new thread
When the start() method is called, the JVM creates a new thread. We represent this new thread object using the instance variable t and start execution.
Print main end...

However, after thread t starts running, main and t run concurrently. At this point, the program itself cannot determine the scheduling order of the threads.

To simulate the effect of concurrent execution, we can call Thread.sleep() in the thread. The parameter unit is milliseconds. sleep() forces the current thread to pause for a while:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


public class Main {
 public static void main(String[] args) {
 System.out.println("main start...");
 Thread t = new Thread() {
 public void run() {
 System.out.println("thread run...");
 try {
 Thread.sleep(10);
 } catch (InterruptedException e) {}
 System.out.println("thread end.");
 }
 };
 t.start();
 try {
 Thread.sleep(20);
 } catch (InterruptedException e) {}
 System.out.println("main end...");
 }
}

Thread Priority

1

Thread.setPriority(int n) // Default is 5

The JVM automatically maps priorities from 1 (lowest) to 10 (highest) to the actual priorities of the OS (different operating systems have different numbers of priority levels). Threads with higher priority are more likely to be scheduled by the OS. The OS might schedule high-priority threads more frequently, but we must never rely on setting priority to guarantee that a high-priority thread executes first. When the CPU is busy, threads with higher priorities acquire more time slices; when the CPU is idle, setting priorities is essentially useless.

The yield() method makes the running thread switch to the ready state, re-contending for the CPU’s time slice. Whether it gets the time slice when contending depends on the CPU’s allocation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


public static native void yield();

Runnable r1 = () -> {
 int count = 0;
 for (;;){
 log.info("---- 1>" + count++);
 }
};
Runnable r2 = () -> {
 int count = 0;
 for (;;){
 Thread.yield();
 log.info(" ---- 2>" + count++);
 }
};
Thread t1 = new Thread(r1,"t1");
Thread t2 = new Thread(r2,"t2");
t1.start();
t2.start();

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


// Execution results
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129504
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129505
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129506
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129507
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129508
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129509
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129510
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129511
11:49:15.796 [t1] INFO thread.TestYield - ---- 1>129512
11:49:15.798 [t2] INFO thread.TestYield - ---- 2>293
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129513
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129514
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129515
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129516
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129517
11:49:15.798 [t1] INFO thread.TestYield - ---- 1>129518

As the results above display, since thread t2 executed yield() every time it ran, the execution opportunities for thread 1 were noticeably more numerous than for thread 2.

Summary

Java uses a Thread object to represent a thread and starts a new thread by calling start().
A thread object can only call the start() method once.
The execution code of a thread is written in the run() method.
Thread scheduling is determined by the OS; the program itself cannot dictate the scheduling sequence.
Thread.sleep() can pause the current thread for a duration.

Thread Blocking

The ways a thread can be placed into a blocking state are as follows:

BIO blocking, i.e., using blocking IO streams.
sleep(long time) forces the thread to sleep, entering the block state.
a.join() invoking thread enters blocking and awaits thread a to finish execution before resuming.
synchronized or ReentrantLock causes a thread to enter the blocking state if it cannot acquire the lock.
Calling the wait() method after acquiring a lock also forces the thread into a blocking state.
LockSupport.park() places the thread in a blocking state.

`Thread.sleep()`

Makes a thread sleep, putting a running thread into a blocked state. When the sleep duration ends, the thread re-contends for the CPU’s time slice to resume execution.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


// Method declaration, a native method
public static native void sleep(long millis) throws InterruptedException; 

try {
 // Sleep for 2 seconds
 // This method throws an InterruptedException, meaning it can be interrupted during sleep, throwing an exception once interrupted
 Thread.sleep(2000);
 } catch (InterruptedException e) {
 }
 try {
 // APIs utilizing TimeUnit serve as a replacement for Thread.sleep 
 TimeUnit.SECONDS.sleep(1);
 } catch (InterruptedException e) {
 }

`Thread.join()`

A thread can also wait for another thread until it concludes its execution. For example, after initiating thread t, the main thread can utilize t.join() to await thread t concluding before continuing to run:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


public class Main {
 public static void main(String[] args) throws InterruptedException {
 Thread t = new Thread(() -> {
 System.out.println("hello");
 });	// Java 8 lambda method
 System.out.println("start");
 t.start(); // Start thread t
 t.join(); // The main thread waits here for t to finish
 System.out.println("end");
 }
}

When the main thread calls join() on thread object t, the main thread will wait until thread t finishes execution, and only then continue running its own subsequent code. Therefore, the print order of the code above is guaranteed: the main thread prints start first, thread t then prints hello, and finally the main thread prints end.

If thread t has already finished, calling join() on instance t returns immediately. Additionally, the overloaded method join(long) allows you to specify a maximum wait time, after which the thread stops waiting.

Summary

Common ways to block a thread: BIO blocking, sleep(), join(), failing to acquire a lock (synchronized/ReentrantLock), wait(), LockSupport.park().
sleep(): Makes the thread sleep for a specified duration. It can be interrupted during sleep. Using TimeUnit is recommended for better readability.
join(): Makes the current thread wait until the target thread finishes execution. Commonly used to control the order of thread execution.
Blocking and resumption: Once a thread enters a blocked state, it must wait for a specific condition to be met (e.g., sleep time elapsed, lock released, target thread completed) before it can resume execution.

Interrupting a Thread

Interrupting a Thread - Java Tutorial - Liao Xuefeng’s Official Website

If a thread needs to execute a long-running task, it may become necessary to interrupt it. Interrupting a thread means that another thread sends it a signal. Upon receiving this signal, the target thread exits its run() method, allowing it to terminate immediately.

For example, when downloading a 100M file from the network, if the connection is slow and the user clicks ‘Cancel’, the program must interrupt the downloading thread.

`Thread.interrupt`

Interrupting a thread is very simple; you just need another thread to call the interrupt() method on the target thread. The target thread needs to repeatedly check whether its state is interrupted, and if so, it must terminate its execution immediately.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


public class Main {
 public static void main(String[] args) throws InterruptedException {
 Thread t = new MyThread();
 t.start();
 Thread.sleep(1); // Pause for 1 millisecond
 t.interrupt(); // Interrupt thread t
 t.join(); // Wait for thread t to end
 System.out.println("end");
 }
}

class MyThread extends Thread {
 public void run() {
 int n = 0;
 while (! isInterrupted()) {
 n++;
 System.out.println(n + " hello!");
 }
 }
}

In the code above, the main thread interrupts thread t by calling t.interrupt(). However, note that the interrupt() method only sends an ‘interrupt request’ to thread t. As for whether thread t can respond immediately, it depends on its code. Since the while loop of thread t detects isInterrupted(), the code above correctly responds to the interrupt() request, allowing the run() method to conclude.

If the thread is in a waiting state—for example, t.join() places the main thread in a waiting state—then if interrupt() is called on the main thread, the join() method will immediately throw an InterruptedException. Therefore, as long as the target thread catches the InterruptedException thrown by the join() method, it implies that another thread has called the interrupt() method on it, and typically the thread should immediately exit.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38


public class Main {
 public static void main(String[] args) throws InterruptedException {
 Thread t = new MyThread();
 t.start();
 Thread.sleep(1000);
 t.interrupt(); // Interrupt thread t
 t.join(); // Wait for thread t to end
 System.out.println("end");
 }
}

class MyThread extends Thread {
 public void run() {
 Thread hello = new HelloThread();
 hello.start(); // Start the hello thread
 try {
 hello.join(); // Wait for the hello thread to end
 } catch (InterruptedException e) {
 System.out.println("interrupted!");
 }
 hello.interrupt();
 }
}

class HelloThread extends Thread {
 public void run() {
 int n = 0;
 while (!isInterrupted()) {
 n++;
 System.out.println(n + " hello!");
 try {
 Thread.sleep(100);
 } catch (InterruptedException e) {
 break;
 }
 }
 }
}

The main thread notifies thread t to interrupt by calling t.interrupt().
At this point, thread t is waiting inside hello.join(); this method immediately stops waiting and throws an InterruptedException.
Inside thread t, the InterruptedException is caught, preparing the thread to terminate.
Before thread t terminates, it also calls interrupt() on the hello thread to notify it to interrupt.

The `running` Flag

Another common method to interrupt a thread is setting a flag. We normally use a running flag to indicate whether the thread should continue executing. By setting HelloThread.running to false from an external thread, we can make the thread terminate:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


public class Main {
 public static void main(String[] args) throws InterruptedException {
 HelloThread t = new HelloThread();
 t.start();
 Thread.sleep(1);
 t.running = false; // Set flag to false
 }
}

class HelloThread extends Thread {
 public volatile boolean running = true;
 public void run() {
 int n = 0;
 while (running) {
 n ++;
 System.out.println(n + " hello!");
 }
 System.out.println("end!");
 }
}

Notice that the flag boolean running of HelloThread is a variable shared between threads. Shared variables between threads need to be marked with the volatile keyword to ensure that every thread can read the updated value of the variable.

The Purpose of `volatile`

Why declare variables shared across threads with the keyword volatile? This relates to Java’s memory model. Inside the Java virtual machine, variable values are stored in main memory. However, when a thread accesses a variable, it first obtains a copy and saves it in its own working memory. If a thread modifies the value of a variable, the virtual machine will write the modified value back to main memory at some point, but this timing is uncertain!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


// This diagram is really well drawn!
┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
 Main Memory
│ │
 ┌───────┐┌───────┐┌───────┐
│ │ var A ││ var B ││ var C │ │
 └───────┘└───────┘└───────┘
│ │ ▲ │ ▲ │
 ─ ─ ─│─│─ ─ ─ ─ ─ ─ ─ ─│─│─ ─ ─
 │ │ │ │
┌ ─ ─ ┼ ┼ ─ ─ ┐ ┌ ─ ─ ┼ ┼ ─ ─ ┐
 ▼ │ ▼ │
│ ┌───────┐ │ │ ┌───────┐ │
 │ var A │ │ var C │
│ └───────┘ │ │ └───────┘ │
 Thread 1 Thread 2
└ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ┘

This causes a situation where if one thread updates a certain variable, the value read by another thread might still be the one before the update.

For example, if the variable in main memory is a = true, when Thread 1 executes a = false, it merely changes its copy of variable a to false at that moment; the variable a in main memory is still true. Before the JVM writes the modified a back to main memory, the value of a read by other threads remains true, which leads to inconsistency in shared variables among multiple threads.

Therefore, the purpose of the volatile keyword is to tell the virtual machine:

Every time you access a variable, always acquire the latest value from main memory;
Every time you modify a variable, instantly write it back to main memory.

The volatile keyword solves the visibility problem: when one thread modifies the value of a shared variable, other threads can immediately see the modified value.

If we remove the volatile keyword and run the program above, we find the effect is similar to having volatile. This is because, under the x86 architecture, the JVM writes back to main memory extremely fast, but switching to an ARM architecture would incur significant delays.

Summary

Calling interrupt() on a target thread sends an interruption request. The target thread checks its status via isInterrupted(). If the target thread is in a waiting state, it will catch an InterruptedException.
A target thread should terminate immediately when isInterrupted() returns true or when it catches an InterruptedException.
When using flag-based approaches to control threads, the volatile keyword must be applied correctly.
The volatile keyword solves the visibility problem of shared variables across threads.

Thread State

Detailed Illustration of Java Multithreading - Personal Article - SegmentFault

Thread State - Java Tutorial - Liao Xuefeng’s Official Website

The System - Five States

Thread states can be divided into five states at the operating system level:

Initial state: The state when the thread object is created.
Runnable state (Ready state): After calling the start() method, it enters the ready state, which implies it’s prepared to be scheduled and executed by the CPU.
Running state: The thread obtains the CPU’s time slice and executes the logic of the run() method.
Blocked state: The thread is blocked, relinquishing the CPU’s time slice, and waits for the block to be lifted to return to the ready state and contend for a time slice again.
Terminated state: The state after the thread has finished execution or thrown an exception.

Java - Six States

In a Java program, a thread object can only call the start() method once to initiate a new thread, and it executes the run() method within the new thread. Once the run() method finishes executing, the thread concludes. Hence, the states of a Java thread are as follows:

NEW: The thread object is created.
Runnable: The thread enters this state after calling the start() method. This state encompasses three scenarios:
1. Ready state: Waiting for the CPU to allocate a time slice.
2. Running state: Entering the Runnable method to execute a task.
3. Blocked state: State during BIO execution of blocking IO streams.
Blocked: The blocked state when failing to acquire a lock (will be detailed in the synchronization lock section).
WAITING: The state after calling methods like wait() or join().
TIMED_WAITING: The state after calling methods like sleep(time), wait(time), or join(time).
TERMINATED: The state after the thread has finished executing or thrown an exception.

After a thread starts, it can switch among the Runnable, Blocked, Waiting, and Timed Waiting states until it finally transitions to the Terminated state, at which point the thread terminates.

The reasons for a thread to terminate include:

Normal termination: The run() method executes and returns at the return statement;
Unexpected termination: The run() method terminates due to an uncaught exception;
Forceful termination: Calling the stop() method on a specific Thread instance (strongly discouraged).

Core Methods in the Thread Class

Method Name	Is Static	Description
start()	No	Starts the thread, entering the ready state to await the CPU allocating a time slice.
run()	No	The method overriding the Runnable interface, representing the specific logic executed when the thread receives a CPU time slice.
yield()	Yes	Thread concession. Forces the thread holding the CPU time slice to enter the ready state to recompete for a time slice.
sleep(time)	Yes	The thread sleeps for a fixed period and enters a blocked state. Once the sleep duration completes, it recompetes for a time slice. Sleeping can be interrupted.
join()/join(time)	No	Calling the `join` method on a thread object forces the calling thread into a blocked state. It waits until the thread object finishes executing or reaches the designated time limit before recovering and re-contending for a time slice.
isInterrupted()	No	Retrieves the thread’s interruption flag: true for interrupted, false for uninterrupted. Calling this will not modify the interruption flag.
interrupt()	No	Interrupts the thread. Methods throwing an `InterruptedException` can all be interrupted; however, after interruption, the flag will not be modified. If a normally executing thread is interrupted, the interruption flag will be modified.
interrupted()	No	Fetches the thread’s interrupted flag. Calling this clears the interruption flag.
stop()	No	Stops thread execution (Not recommended).
suspend()	No	Suspends thread (Not recommended).
resume()	No	Resumes thread execution (Not recommended).
currentThread()	Yes	Acquires the current thread.

Thread-related methods in Object

Method Name	Description
wait()/wait(long timeout)	Makes the thread that has acquired the lock enter a blocked state.
notify()	Randomly wakes up one thread that has been `wait()`-ed.
notifyAll();	Wakes up all threads that have been `wait()`-ed so they can recompete for time slices.

Daemon Threads

Daemon Threads - Java Tutorial - Liao Xuefeng’s Official Website

The Java program entry point involves the JVM launching the main thread, which can in turn launch other threads. When all threads have finished executing, the JVM exits, and the process ends.

If there is a thread that hasn’t exited, the JVM process will not exit. Therefore, it must be guaranteed that all threads can conclude promptly.

However, there is a type of thread whose purpose is looping unconditionally. For example, a thread that triggers a task on a timer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


class TimerThread extends Thread {
 @Override
 public void run() {
 while (true) {
 System.out.println(LocalTime.now());
 try {
 Thread.sleep(1000);
 } catch (InterruptedException e) {
 break;
 }
 }
 }
}

If this thread does not finish, the JVM process cannot end. The question is, who is responsible for closing this thread?

Often, such threads lack a designated manager to terminate them. However, when other threads have finished, the JVM process unequivocally must end. What can be done?

The answer is using a Daemon Thread.

A daemon thread refers to a thread that serves other threads. In the JVM, once all non-daemon threads have completed execution, regardless of whether daemon threads exist, the virtual machine will automatically exit.

Hence, when the JVM exits, it doesn’t need to care whether daemon threads have concluded.

How does one create a daemon thread? The method is identical to an ordinary thread; only, before calling the start() method, you call setDaemon(true) to mark the thread as a daemon thread:

1
2
3


Thread t = new MyThread();
t.setDaemon(true);
t.start();

Inside a daemon thread, caution must be exercised when writing code: Daemon threads cannot hold any resources that require closing, such as opened files. This is because when the virtual machine exits, the daemon thread is afforded no opportunity to close the files, which will result in data loss.

Summary

Daemon threads are threads that serve other threads.

After all non-daemon threads have completed execution, the virtual machine exits, and daemon threads are consequently terminated.

Daemon threads cannot hold resources that require closure (e.g., opened files).

Thread Synchronization

Thread Synchronization - Java Tutorial - Liao Xuefeng’s Official Website

When multiple threads execute concurrently, the scheduling of threads is dictated by the operating system, and the program itself cannot control it. Consequently, there is a possibility for any thread to be paused by the OS at any instruction and then resume execution after a certain timeframe.

At this point, a problem emerges that doesn’t exist under single-threaded models: if multiple threads concurrently read and write to a shared variable, data inconsistency issues will arise.

Let’s look at an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


// Multiple threads
public class Main {
 public static void main(String[] args) throws Exception {
 var add = new AddThread();
 var dec = new DecThread();
 add.start();
 dec.start();
 add.join();
 dec.join();
 System.out.println(Counter.count);
 }
}

class Counter {
 public static int count = 0;
}

class AddThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) { Counter.count += 1; }
 }
}

class DecThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) { Counter.count -= 1; }
 }
}

The code above is very simple. Two threads simultaneously perform operations on an int variable; one adds 1 ten thousand times, and the other subtracts 1 ten thousand times. Ultimately, the result should be 0. However, every time it runs, the actual result varies.

This is because when reading and writing a variable, to get the correct result, it must be guaranteed to be an atomic operation. Atomic operations are single operations or a sequence of operations that cannot be interrupted.

For example, regarding the statement:

1

n = n + 1;

It appears to be a single statement, but in reality, it maps to 3 instructions:

1
2
3


ILOAD
IADD
ISTORE

Suppose the value of n is 100. If two threads concurrently execute n = n + 1, the obtained result is highly likely not 102, but rather 101. The reason being:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


┌───────┐ ┌───────┐
│Thread1│ │Thread2│
└───┬───┘ └───┬───┘
 │ │
 │ILOAD (100) │
 │ │ILOAD (100)
 │ │IADD
 │ │ISTORE (101)
 │IADD │
 │ISTORE (101) │
 ▼ ▼

If Thread 1 is interrupted by the OS after executing ILOAD, and if Thread 2 is scheduled to run at that exact moment, the value it retrieves after executing ILOAD is still 100. Ultimately, after the ISTORE writes of both threads, the result becomes 101 instead of the anticipated 102.

This demonstrates that beneath the multithreaded model, to ensure logic exactness, when reading and writing shared variables, you must ensure a group of instructions are executed atomically: meaning when an individual thread is executing, other threads must wait:

`synchronized` Synchronization Lock

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


┌───────┐ ┌───────┐
│Thread1│ │Thread2│
└───┬───┘ └───┬───┘
 │ │
 │-- lock -- │
 │ILOAD (100) │
 │IADD │
 │ISTORE (101) │
 │-- unlock -- │
 │ │-- lock --
 │ │ILOAD (101)
 │ │IADD
 │ │ISTORE (102)
 │ │-- unlock --
 ▼ ▼

Through lock and unlock operations, we ensure that the 3 instructions always execute within a single thread’s execution period, preventing other threads from entering this instruction region. Even if the executing thread is interrupted by the OS, other threads still cannot enter this region because they cannot acquire the lock. Only after the executing thread releases the lock can other threads acquire it and proceed. The code block between locking and unlocking is called a Critical Section. At any given time, at most one thread can execute within the critical section.

Evidently, ensuring the atomicity of a segment of code is achieved by acquiring and releasing a lock. A Java program uses the synchronized keyword to lock an object:

1
2
3


synchronized(lock) {
 n = n + 1;
}

synchronized guarantees that the code block can be executed by at most one thread at an arbitrary moment. We can rewrite the code above utilizing synchronized as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


// Multiple threads
public class Main {
 public static void main(String[] args) throws Exception {
 var add = new AddThread();
 var dec = new DecThread();
 add.start();
 dec.start();
 add.join();
 dec.join();
 System.out.println(Counter.count);
 }
}

class Counter {
 public static final Object lock = new Object();
 public static int count = 0;
}

class AddThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lock) {
 Counter.count += 1;
 }
 }
 }
}

class DecThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lock) {
 Counter.count -= 1;
 }
 }
 }
}

Observe the code:

1
2
3


synchronized(Counter.lock) { // Acquire lock
 ...
} // Release lock

It indicates using the Counter.lock instance as a lock. When the two threads execute their respective synchronized(Counter.lock) { ... } code blocks, they must first acquire the lock before entering the code block. After execution concludes, the lock is automatically released at the end of the synchronized statement block. In this way, reading and writing the Counter.count variable simultaneously is impossible. No matter how many times the code above is run, the final result is always 0.

Using synchronized solves the problem of correct concurrent access to shared variables by multiple threads. However, its disadvantage is a performance drop, because synchronized code blocks cannot execute concurrently. Additionally, acquiring and releasing locks requires a certain amount of time, meaning synchronized reduces the program’s execution efficiency.

Let’s outline how to use synchronized:

Identify the thread code blocks that modify shared variables;
Choose a shared instance as a lock;
Use synchronized(lockObject) { ... }.

When using synchronized, you do not need to worry about exceptions being thrown. Because regardless of whether there is an exception or not, the lock will be released correctly at the end of synchronized:

1
2
3
4
5
6
7
8


public void add(int m) {
 synchronized (obj) {
 if (m < 0) {
 throw new RuntimeException();
 }
 this.value += m;
 } // The lock is released here regardless of exceptions
}

Moreover, multiple threads can concurrently obtain their respective locks simultaneously: because JVM only ensures that the same lock can only be acquired by one thread at any arbitrary moment, but two different locks can be acquired separately by two threads at the same time.

Therefore, when using synchronized, which lock is acquired is extremely important. If the lock object is incorrect, the code logic will be wrong.

Below is an example of employing two different locks to improve efficiency:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60


public class Main {
 public static void main(String[] args) throws Exception {
 var ts = new Thread[] { new AddStudentThread(), new DecStudentThread(), new AddTeacherThread(), new DecTeacherThread() };
 for (var t : ts) {
 t.start();
 }
 for (var t : ts) {
 t.join();
 }
 System.out.println(Counter.studentCount);
 System.out.println(Counter.teacherCount);
 }
}

class Counter {
 public static final Object lockStudent = new Object();
 public static final Object lockTeacher = new Object();
 public static int studentCount = 0;
 public static int teacherCount = 0;
}

class AddStudentThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lockStudent) {
 Counter.studentCount += 1;
 }
 }
 }
}

class DecStudentThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lockStudent) {
 Counter.studentCount -= 1;
 }
 }
 }
}

class AddTeacherThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lockTeacher) {
 Counter.teacherCount += 1;
 }
 }
 }
}

class DecTeacherThread extends Thread {
 public void run() {
 for (int i=0; i<10000; i++) {
 synchronized(Counter.lockTeacher) {
 Counter.teacherCount -= 1;
 }
 }
 }
}

Operations That Do Not Require `synchronized`

The JVM specification defines several atomic operations:

Assignment of basic types (excluding long and double), e.g., int n = m;
Reference type assignment, e.g., List<String> list = anotherList.

long and double are 64-bit data. The JVM does not strictly specify whether 64-bit assignments are atomic, but on x64 platform JVMs, the assignments of long and double are implemented as atomic operations.

Statements with a single atomic operation do not require synchronization. For example:

1
2
3
4
5


public void set(int m) {
 synchronized(lock) {
 this.value = m;
 }
}

Does not require synchronization.

It’s similar for references. For example:

1
2
3


public void set(String s) {
 this.value = s;
}

The aforementioned assignment statement does not require synchronization.

However, if they are multi-line assignment statements, they must be guaranteed to be synchronized operations. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


class Point {
 int x;
 int y;
 public void set(int x, int y) {
 synchronized(this) {
 this.x = x;
 this.y = y;
 }
 }
 public int[] get() {
 synchronized(this) {
 return new int[]{x,y};
 }
 }
}

The read and write operations above, namely (set(), get()), need synchronization. If reading is unsynchronized, it will cause logical errors in the program:

1
2
3
4
5


public int[] get() {
 int[] copy = new int[2];
 copy[0] = x;
 copy[1] = y;
 }

Suppose the current coordinates are (100, 200). Then, when setting the new coordinates to (110, 220), the values read multithreadedly by the aforementioned unsynchronized code might be:

(100, 200): before updating x and y;
(110, 200): after updating x, before updating y;
(110, 220): after updating x and y.

If it reads (110, 200), i.e., having read the updated x but the pre-update y, there’s no guarantee the states of multiple read variables stay consistent.

Sometimes, through some clever transformations, non-atomic operations can be turned into atomic operations. For example, if the code above is rewritten as:

1
2
3
4
5
6
7


class Point {
 int[] ps;
 public void set(int x, int y) {
 int[] ps = new int[] { x, y };
 this.ps = ps;
 }
}

Synchronization is no longer required because this.ps = ps is an atomic operation for reference assignment. Meanwhile, the statement:

1

int[] ps = new int[] { x, y };

Here, ps is a local variable defined inside the method. Every thread will have its own individual local variables, unaffecting each other and remaining mutually invisible, hence demanding no synchronization.

Note, however, that the reading method still requires synchronization during the process of copying the int[] array.

Immutable Objects Do Not Require Synchronization

Immutable objects denote objects whose state cannot be altered after creation. In Java, typical immutable objects include:

String
Immutable collections created by List.of() (Java 9+)
Wrapper classes for basic types (e.g., Integer, Long, etc.)

If multiple threads read from or write to an immutable object, synchronization isn’t necessary because the object’s state won’t be modified:

1
2
3
4
5
6
7
8
9


class Data {
 List<String> names;
 void set(String[] names) {
 this.names = List.of(names);
 }
 List<String> get() {
 return this.names;
 }
}

Notice that the set() method internally created an immutable List. The objects incorporated in this List are also immutable String objects; therefore, the whole List<String> instance comprises immutability, making both reading and writing synchrony-free.

When analyzing whether a variable can be accessed concurrently by multiple threads, one must first clarify the concepts. What multiple threads execute simultaneously are methods. For the example below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


class Status {
 List<String> names;
 int x;
 int y;
 void set(String[] names, int n) {
 List<String> ns = List.of(names);
 this.names = ns;
 int step = n * 10;
 this.x += step;
 this.y += step;
 }
 StatusRecord get() {
 return new StatusRecord(this.names, this.x, this.y);
 }
}

If threads A and B exist, “executing simultaneously” signifies:

set() might execute simultaneously;
get() might execute simultaneously;
A might execute set() concurrently while B executes get().

The class member variables names, x, y clearly can be simultaneously read and written by multiple threads, but local variables (including method parameters) if not “escaped”, remain solely visible to the current thread. The local variable step is only used inside the set() method, therefore when every thread executes set(), it contains an independent storage of step in the thread’s stack, without mutual influence.

The local variable ns is also held separately by each thread, but the subsequent assignment this.names = ns turns it visible to other threads. If the set() method is synchronized, and you wish to minimize the synchronized code block, you can rewrite it as:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


void set(String[] names, int n) {
 // Local variables are invisible to other threads:
 List<String> ns = List.of(names);
 int step = n * 10;
 synchronized(this) {
 this.names = ns;
 this.x += step;
 this.y += step;
 }
}

Therefore, deeply understanding multithreading requires comprehending variable storage in the stack, as primitive types and reference types are stored differently.

Scenario	Requires Sync	Reason
Immutable object (e.g., `List.of()`)	No	Object immutable, multi-thread read-only, no race conditions.
Local variable (e.g., `step`)	No	Thread private, confined to stack.
Member variable assignment (e.g., `this.names`)	Yes	Reference could be modified simultaneously; needs sync or `volatile`.
Compound ops (e.g., `x += step`)	Yes	Non-atomic operations (read-modify-write); needs sync.

Summary

When multiple threads concurrently read and write shared variables, logical errors may occur; therefore, synchronization via synchronized is required.

The essence of synchronization is locking a specified object; only after acquiring the lock can the subsequent code execute.

Note that the lock object must be the same instance.

Single atomic operations defined by the JVM do not require synchronization.

Thread Synchronization Methods

Thread Safety

If a class is designed to permit multiple threads to access it correctly, we say this class is “thread-safe.” The java.lang.StringBuffer in the Java standard library is also thread-safe.

There are also some immutable classes, such as String, Integer, LocalDate, whose member variables are all final. Multiple threads can only read and cannot write when accessing them simultaneously. These immutable classes are also thread-safe.

Lastly, classes like Math that only provide static methods and have no member variables are also thread-safe.

Apart from the few exceptions mentioned above, most classes, such as ArrayList, are non-thread-safe classes. We cannot modify them in a multithreaded environment. However, if all threads only read and do not write, then ArrayList can be safely shared across threads.

Without specific elaboration, a class is non-thread-safe by default.

Take the Counter class below for example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


public class Counter {
 private int count = 0;

 public void add(int n) {
 synchronized(this) {
 count += n;
 }
 }

 public void dec(int n) {
 synchronized(this) {
 count -= n;
 }
 }

 public int get() {
 return count;
 }
}

This way, when a thread calls the add() and dec() methods, it doesn’t need to care about synchronization logic because the synchronized code block is inside the add() and dec() methods. Moreover, we notice that the object locked by synchronized is this, meaning the current instance, which again ensures that when multiple Counter instances are created, they do not influence each other and can execute concurrently.

The `synchronized` Modifier

Let’s observe the Counter code again:

1
2
3
4
5
6
7
8


public class Counter {
 public void add(int n) {
 synchronized(this) {
 count += n;
 }
 }
 ...
}

When what we lock is the this instance, we can actually use synchronized to modify the method. The following two approaches are equivalent:

1
2
3
4
5


public void add(int n) {
 synchronized(this) { // Lock 'this'
 count += n;
 } // Unlock
}

Approach two:

1
2
3


public synchronized void add(int n) { // Lock 'this'
 count += n;
} // Unlock

Therefore, a method modified with synchronized is a synchronized method, which signifies that the entire method must be locked using the this instance.

For static methods, there is no this instance because static methods target the class rather than an instance. However, we note that any class has a Class instance automatically created by the JVM. Hence, adding synchronized to a static method locks the Class instance of that class. The aforementioned synchronized static method actually equates to:

1
2
3
4
5
6
7


public class Counter {
 public static void test(int n) {
 synchronized(Counter.class) {
 ...
 }
 }
}

Summary

Using synchronized to modify a method can turn the entire method into a synchronized code block. The locking object for a synchronized method is this.

Through reasonable design and data encapsulation, a class can become “thread-safe”.

Unless otherwise stated, a class is not thread-safe by default.

Whether multiple threads can safely access a certain non-thread-safe instance requires analyzing the specific situation.

Deadlock

Reentrant Locks

Java’s thread locks are reentrant locks.

What is a reentrant lock? Let’s check out an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


public class Counter {
 private int count = 0;

 public synchronized void add(int n) {
 if (n < 0) {
 dec(-n);
 } else {
 count += n;
 }
 }

 public synchronized void dec(int n) {
 count += n;
 }
}

Execution flow:

Calling add(-1):
- Acquires the this lock: counter = 1, holding thread = current thread.
Calls dec(1) after entering the add method:
- Acquires the this lock again: discovers it is already held by the current thread, counter increases to 2.
Exits the dec method:
- Counter decreases to 1.
Exits the add method:
- Counter decreases to 0, lock truly released.

Observe the add() method modified by synchronized. Once a thread executes inside the add() method, it implies that it has already obtained the this lock of the current instance. If the passed n < 0, the dec() method will be called inside the add() method. Because the dec() method also needs to acquire the this lock, a question arises:

For the same thread, is it possible to continue acquiring the same lock after having already acquired it?

The answer is affirmative. The JVM permits the same thread to repeatedly acquire the same lock. A lock that can be repeatedly acquired by the same thread is called a reentrant lock.

Since Java’s thread locks are reentrant locks, when acquiring a lock, it not only checks whether it is being acquired for the first time but also records the number of acquisitions. Each time the lock is acquired, the record is incremented by 1, and each time a synchronized block is exited, the record is decremented by 1. Only when the record decreases to 0 is the lock genuinely released.

Deadlock

A thread can acquire one lock and then proceed to acquire another lock. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


public void add(int m) {
 synchronized(lockA) { // Acquire the lock for lockA
 this.value += m;
 synchronized(lockB) { // Acquire the lock for lockB
 this.another += m;
 } // Release the lock for lockB
 } // Release the lock for lockA
}

public void dec(int m) {
 synchronized(lockB) { // Acquire the lock for lockB
 this.another -= m;
 synchronized(lockA) { // Acquire the lock for lockA
 this.value -= m;
 } // Release the lock for lockA
 } // Release the lock for lockB
}

When acquiring multiple locks, different threads acquiring locks of multiple distinct objects may induce a deadlock. For the code above, if thread 1 and thread 2 simultaneously execute the add() and dec() methods respectively:

Thread 1: Enters add(), obtains lockA;
Thread 2: Enters dec(), obtains lockB.

Subsequently:

Thread 1: Prepares to obtain lockB, fails, waiting;
Thread 2: Prepares to obtain lockA, fails, waiting.

At this point, the two threads each hold different locks and then attempt to acquire the lock held by the other, resulting in an infinite mutual wait. This is a deadlock.

After a deadlock occurs, there’s no mechanism to clear it; the JVM process can merely be forcefully terminated.

Hence, when writing multi-threaded applications, particular attention should be paid to guard against deadlock. Because once a deadlock forms, one can only forcefully terminate the process.

So how should we avoid deadlocks? The answer is: the order in which threads acquire locks must be consistent. Specifically, strictly follow the order of acquiring lockA first, then lockB. The rewritten dec() method is as follows:

1
2
3
4
5
6
7
8


public void dec(int m) {
 synchronized(lockA) { // Acquire the lock for lockA
 this.value -= m;
 synchronized(lockB) { // Acquire the lock for lockB
 this.another -= m;
 } // Release the lock for lockB
 } // Release the lock for lockA
}

Summary

Java’s synchronized locks are reentrant locks.

Deadlock preconditions imply multiple threads each hold different locks and mutually attempt to retrieve the locks already held by the other, causing infinite waiting.

Avoiding deadlock relies on multiple threads acquiring locks in an identical order.

Thread Communication

In Java programs, synchronized resolves the problem of multithread competition. For instance, for a task manager, when multiple threads concurrently add tasks to a queue, synchronized can be used to apply locks:

1
2
3
4
5
6
7


class TaskQueue {
 Queue<String> queue = new LinkedList<>();

 public synchronized void addTask(String s) {
 this.queue.add(s);
 }
}

However, synchronized does not solve the coordination problem of multiple threads.

Still using the TaskQueue above as an example, let’s write another getTask() method to extract the first task from the queue:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


class TaskQueue {
 Queue<String> queue = new LinkedList<>();

 public synchronized void addTask(String s) {
 this.queue.add(s);
 }

 public synchronized String getTask() {
 while (queue.isEmpty()) {
 }
 return queue.remove();
 }
}

The code above seems faultless: getTask() initially checks whether the queue is empty internally. If it is empty, it waits in a loop until another thread inserts a task into the queue. The while() loop exits, and it can return the element from the queue.

However, the while() loop will never actually exit. Because when the thread executes the while() loop, it has already acquired the this lock at the entrance of getTask(). Other threads can’t possibly call addTask(), as executing addTask() also requires acquiring the this lock.

Therefore, executing the code above will cause the thread to 100% consume CPU resources inside getTask() due to an infinite loop.

If we think deeper, the execution effect we desire is:

Thread 1 can call addTask() to constantly add tasks to the queue;
Thread 2 can call getTask() to fetch tasks from the queue. If the queue is empty, getTask() should wait until there is at least one task in the queue before returning.

Thus, the principle of multiple threads coordinating their execution is: when conditions are not met, the thread enters a waiting state; when conditions are met, the thread is awakened to continue executing tasks.

`wait()`

For the TaskQueue above, let’s first transform the getTask() method to make the thread enter a waiting state when conditions aren’t met:

1
2
3
4
5
6


public synchronized String getTask() {
 while (queue.isEmpty()) {
 this.wait();
 }
 return queue.remove();
}

When a thread executes to the while loop interior of the getTask() method, it must have already acquired the this lock. At this point, the thread evaluates the while condition. If the condition holds true (queue is empty), the thread will execute this.wait(), entering a waiting state.

The key here is: the wait() method must be invoked on the lock object currently acquired. The lock acquired here is this, hence the call to this.wait().

After a thread invokes wait(), it enters a waiting state. The wait() method won’t return until a subsequent moment when the thread is awakened from its waiting state by another thread. After yielding to wait(), the thread continues processing the next statement.

Some diligent folks might point out: even if a thread rests inside getTask(), if other threads fail to snag the this lock, they still won’t be able to execute addTask(), what do we do?

The crux of this problem lies in the fact that the execution mechanism of wait() is highly complex. First, it’s not a regular Java method, but a native method defined in the Object class, meaning it is implemented by JVM’s C code. Secondly, the wait() method can only be invoked within a synchronized block, because when wait() is called, it will release the lock obtained by the thread. When wait() returns, the thread will again attempt to acquire the lock.

Therefore, the wait() method can only be invoked on the object lock. Because we acquired the this lock in getTask(), the wait() method can only be called on the this object.

1
2
3
4
5
6
7
8


public synchronized String getTask() {
 while (queue.isEmpty()) {
 // Release the 'this' lock:
 this.wait();
 // Reacquire the 'this' lock
 }
 return queue.remove();
}

When a thread sleeps at this.wait(), it yields the this lock, enabling other threads to snare the this lock inside the addTask() method.

`notify()`

Now we face a second problem: how do we get the slumbering thread to be reawakened and then return from wait()? The answer is calling notify() on the same lock object. Let’s alter addTask() as follows:

1
2
3
4


public synchronized void addTask(String s) {
 this.queue.add(s);
 this.notify(); // Wake up threads waiting on the 'this' lock
}

Notice that immediately after lodging a task into the queue, the thread instantly calls notify() on the this lock object. This method will awaken a thread that is presently sleeping on the this lock (which is the sequence suspended at this.wait() interior to getTask()), rendering the suspended thread capable of returning from this.wait().

Let’s scrutinize a full example (which is also a producer-consumer model):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56


import java.util.*;

public class Main {
 public static void main(String[] args) throws InterruptedException {
 var q = new TaskQueue();
 var ts = new ArrayList<Thread>();
 for (int i=0; i<5; i++) {
 var t = new Thread() {
 public void run() {
 // Execute task:
 while (true) {
 try {
 String s = q.getTask();
 System.out.println("execute task: " + s);
 } catch (InterruptedException e) {
 return;
 }
 }
 }
 };
 t.start();
 ts.add(t);
 }
 var add = new Thread(() -> {
 for (int i=0; i<10; i++) {
 // Insert task:
 String s = "t-" + Math.random();
 System.out.println("add task: " + s);
 q.addTask(s);
 try { Thread.sleep(100); } catch(InterruptedException e) {}
 }
 });
 add.start();
 add.join();
 Thread.sleep(100);
 for (var t : ts) {
 t.interrupt();
 }
 }
}

class TaskQueue {
 Queue<String> queue = new LinkedList<>();

 public synchronized void addTask(String s) {
 this.queue.add(s);
 this.notifyAll();
 }

 public synchronized String getTask() throws InterruptedException {
 while (queue.isEmpty()) {
 this.wait();
 }
 return queue.remove();
 }
}

In this example, our focus is the addTask() method, which calls this.notifyAll() instead of this.notify(). Exerting notifyAll() awakens all threads presently lingering at the this lock, while notify() solely stirs one of them (which exact thread is contingent on the operating system, carrying explicit randomness). Because multiple threads might be waiting inside the wait() of the getTask() method, using notifyAll() will awaken them all at once. Usually speaking, notifyAll() is much safer. At times, if the logic lacks comprehensiveness, using notify() might lead to only one thread waking, while others might wait perpetually, never to wake.

Still, take heed that wait() necessitates reobtaining the this lock when it returns. Suppose 3 threads receive a wake-up signal; following the awakening, they must first wait for the thread executing addTask() to finish the method before the this lock is dropped. Subsequently, among these 3 threads, only one manages to grasp the this lock; the leftover two will plunge back into waiting.

Also observe how we deploy wait() within a while() loop rather than within an if block:

1
2
3
4
5
6


public synchronized String getTask() throws InterruptedException {
 if (queue.isEmpty()) {
 this.wait();
 }
 return queue.remove();
}

This arrangement fundamentally harbors errors, since a thread necessitates reacquiring the this lock after awakening. Once several threads are spurred, merely one thread nabs the this lock. Currently this thread executing queue.remove() extracts elements correctly, however, when the remaining threads attain the this lock down the line and execute queue.remove(), the queue might no longer contain any elements. Therefore wait() should always reside in a while loop, and the lock must be rechecked explicitly upon acquisition.

1
2
3


while (queue.isEmpty()) {
 this.wait();
}

Summary

wait and notify are employed for multithread coordination:

Within synchronized, invoking wait() drives a thread into a waiting state;
wait() must be triggered from an already held lock object;
Within synchronized, notify() or notifyAll() can be called to awaken other waiting threads;
notify() or notifyAll() must be triggered on an already-grasped lock object;
Awakened threads are still compelled to secure the lock anew before they proceed with execution.

Producer-Consumer Model

Java Producer Consumer Model Implementation and Analysis_bilibili

Below is a simple producer-consumer model example from Bilibili. Although it’s not as complex as the preceding thread communication example or the messaging queue example below, mastering these three examples should be sufficient to grasp this model.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


public class Demo1 {
 /**
 * Alternates execution of two threads
 * One outputs "1,2,3,..."
 * The other outputs "a,b,c,..."
 */
 public static void main(String[] args) {
 Factory factory = new Factory();

 final Thread t1 = new Thread(new Runnable() {
 @Override
 public void run() {
 for(int i = 1;i <= 26;i++){
 factory.product(i);
 }
 }
 });

 final Thread t2 = new Thread(new Runnable() {
 @Override
 public void run() {
 for(int i = 'a';i <= 'z';i++){
 factory.consume((char) i);
 }
 }
 });
 t1.start();
 t2.start();
 }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34


public class Factory {
 /**
 * 0: Producer is producing, consumer is waiting. After producing, producer notifies consumer to consume.
 * 1: Consumer is consuming, producer is waiting. After consuming, consumer notifies producer to produce.
 */
 private int sign = 0;	// State value


 public synchronized void product(int n){
 if(sign == 1){
 try {
 this.wait();
 } catch (InterruptedException e) {
 e.printStackTrace();
 }
 }
 System.out.print(n);
 this.notify();
 this.sign = 1;
 }

 public synchronized void consume(char c){
 if(sign == 0){
 try {
 this.wait();
 } catch (InterruptedException e) {
 e.printStackTrace();
 }
 }
 System.out.print(c);
 this.notify();
 this.sign = 0;
 }
}

The execution of threads carries a degree of randomness that users cannot completely control. However, the producer-consumer model can achieve the “alternating” execution of two threads.

The comments explain most of the logic. Let’s analyze it further:

Assume thread t1 is called first. Since sign = 0, it prints character 1 and changes sign to 1. From here, there are two possibilities: thread t1 or t2 is called next.

Calling t1:

sign = 1, enters the try/catch block.
Calls this.wait() on the synchronization lock object, entering the “waiting” state.
wait() will release the lock. Thread t2 executes, running consume().
notify() awakens thread t1 currently waiting on this.
sign is assigned 0, and the cycle repeats.

Calling t2:

Thread t2 executes, running consume().
notify() does not awaken any thread (because no thread is in a waiting state).
sign is assigned 0, and the cycle repeats.

Example Analysis

Below is a more complex (and realistic) example. The concept is quite similar to the simple example above.

Let’s clarify the inline lambda expression used to create a thread in the example below:

new Thread() - Creates a new thread

() -> {...} - Lambda expression defining the thread task

"Producer" + i - Thread naming

.start() - Starts the thread

Here, threads are created using a loop, so the loop variable is used to name them.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75


 public static void main(String[] args) throws InterruptedException {
 MessageQueue queue = new MessageQueue(2);

 // Three producers put values into the queue
 for (int i = 0; i < 3; i++) {
 int id = i;
 new Thread(() -> {
 queue.put(new Message(id, "Value " + id));
 }, "Producer " + i).start();
 }

 Thread.sleep(1000);

 // One consumer continuously takes values from the queue
 new Thread(() -> {
 while (true) {
 queue.take();
 }
 }, "Consumer").start();

 }
}

// Message queue is shared by producers and consumers
class MessageQueue {
 private LinkedList<Message> list = new LinkedList<>();

 // Capacity
 private int capacity;

 public MessageQueue(int capacity) {
 this.capacity = capacity;
 }
 // Producer
 public void put(Message message) {
 synchronized (list) {
 while (list.size() == capacity) {
 log.info("Queue is full, producer is waiting");
 try {
 list.wait();
 } catch (InterruptedException e) {
 e.printStackTrace();
 }
 }
 list.addLast(message);
 log.info("Produced message: {}", message);
 // Notify consumers after producing
 list.notifyAll();
 }
 }
 // Consumer
 public Message take() {
 synchronized (list) {
 while (list.isEmpty()) {
 log.info("Queue is empty, consumer is waiting");
 try {
 list.wait();
 } catch (InterruptedException e) {
 e.printStackTrace();
 }
 }
 Message message = list.removeFirst();
 // Retrieve message from the head of the queue
 log.info("Consumed message: {}", message);
 // Notify producers after consuming
 list.notifyAll();
 return message;
 }
 }
}
 // Message
class Message {
 private int id;
 private Object value;
}

Main Function:

Creates a MessageQueue with a capacity of 2.
Starts 3 producer threads. Each producer puts one message into the queue.
The main thread sleeps for 1 second, giving producers enough time to start working.
Starts a consumer thread that continuously extracts messages from the queue.

Producer:

Uses a synchronized block to acquire the lock for the list object.
Checks if the queue is full (using a while loop to prevent spurious wakeups).
If the queue is full, calls wait() to release the lock and wait.
When there is free space in the queue, adds a message to the end of the queue.
Calls notifyAll() to wake up any consumer threads that might be waiting.

Consumer:

Uses a synchronized block to acquire the lock for the list object.
Checks if the queue is empty (using a while loop to prevent spurious wakeups).
If the queue is empty, calls wait() to release the lock and wait.
When there are messages in the queue, retrieves a message from the head of the queue.
Calls notifyAll() to wake up any producer threads that might be waiting.
Returns the retrieved message.

Summary

Synchronization Mechanism: Uses synchronized to ensure atomic operations on the queue.
Wait/Notify Mechanism: Uses wait() and notifyAll() to achieve inter-thread communication.
Loop Condition Check: Uses while instead of if to check conditions, preventing spurious wakeups.
Capacity Limits: Controls queue size to prevent memory exhaustion.

ReentrantLock

Starting with Java 5, an advanced java.util.concurrent package was introduced to handle concurrency. It provides numerous robust concurrent functionalities, greatly simplifying multi-threaded programming.

We know that Java natively provides the synchronized keyword for locking. However, this lock is somewhat heavy, and when trying to acquire it, threads must wait indefinitely without any mechanism to attempt locking and abort if failed.

The ReentrantLock offered by the java.util.concurrent.locks package serves as a substitute for synchronized locking. Let’s look at conventional synchronized code:

1
2
3
4
5
6
7
8
9


public class Counter {
 private int count;

 public void add(int n) {
 synchronized(this) {
 count += n;
 }
 }
}

If we replace it with ReentrantLock, we can modify the code as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


public class Counter {
 private final Lock lock = new ReentrantLock();
 private int count;

 public void add(int n) {
 lock.lock();
 try {
 count += n;
 } finally {
 lock.unlock();
 }
 }
}

Because synchronized is syntax provided directly at the Java language level, we don’t consider exceptions. But since ReentrantLock is a lock implemented in Java code, we must explicitly acquire the lock and then reliably release it within a finally block.

As the name implies, ReentrantLock is a reentrant lock. Like synchronized, a thread can acquire the same lock multiple times.

Unlike synchronized, ReentrantLock allows one to attempt acquiring a lock:

1
2
3
4
5
6
7


if (lock.tryLock(1, TimeUnit.SECONDS)) {
 try {
 ...
 } finally {
 lock.unlock();
 }
}

In the code above, when trying to capture the lock, it waits up to 1 second. If the lock is still not obtained after 1 second, tryLock() returns false. This allows the program to handle it elegantly rather than waiting infinitely.

Therefore, using ReentrantLock is safer than raw synchronized; if a thread fails during tryLock(), it won’t lead to a deadlock.

Below, we introduce its various methods, along with a more complex example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


// Default non-fair lock, passing 'true' creates a fair lock
ReentrantLock lock = new ReentrantLock(false);
// Try to acquire the lock
lock()
// Release the lock. Should be placed in a finally block to ensure it is executed
unlock()
try {
 // Can be interrupted while acquiring the lock; blocked threads can be interrupted
 LOCK.lockInterruptibly();
} catch (InterruptedException e) {
 return;
}
// Try to acquire the lock. Returns false if unobtainable
LOCK.tryLock()
// Supports timeout. Returns false if the lock is not acquired within the specified duration
tryLock(long timeout, TimeUnit unit)
// Specify condition variable (waiting room). One lock can create multiple waiting rooms
Condition waitSet = ROOM.newCondition();
// Release the lock and enter waitSet to wait. Other threads can contend for the lock after release
yanWaitSet.await()
// Wake up threads in a specific waiting room. After waking up, they re-compete for the lock
yanWaitSet.signal()

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67


 public static void main(String[] args) {
 AwaitSignal awaitSignal = new AwaitSignal(5);
 // Build three condition variables
 Condition a = awaitSignal.newCondition();
 Condition b = awaitSignal.newCondition();
 Condition c = awaitSignal.newCondition();
 // Start three threads
 new Thread(() -> {
 awaitSignal.print("a", a, b);
 }).start();

 new Thread(() -> {
 awaitSignal.print("b", b, c);
 }).start();

 new Thread(() -> {
 awaitSignal.print("c", c, a);
 }).start();

 try {
 Thread.sleep(1000);
 } catch (InterruptedException e) {
 e.printStackTrace();
 }
 awaitSignal.lock();
 try {
 // Wake up 'a' first
 a.signal();
 } finally {
 awaitSignal.unlock();
 }
 }

}

class AwaitSignal extends ReentrantLock {

 // Number of loops
 private int loopNumber;

 public AwaitSignal(int loopNumber) {
 this.loopNumber = loopNumber;
 }

 /**
 * @param print Character to print
 * @param current Current condition variable
 * @param next Next condition variable
 */
 public void print(String print, Condition current, Condition next) {

 for (int i = 0; i < loopNumber; i++) {
 lock();
 try {
 try {
 // Wait after acquiring the lock
 current.await();
 System.out.print(print);
 } catch (InterruptedException e) {
 
 }
 next.signal();
 } finally {
 unlock();
 }
 }
 }

Process Analysis:

Initialization:
The main thread creates an AwaitSignal object, setting the number of loops to 5.
Three Condition objects are created: a, b, c, corresponding respectively to three threads.
The three threads start, respectively calling print("a", a, b), print("b", b, c), and print("c", c, a).
After sleeping for 1 second, the main thread acquires the lock and wakes up thread A via a.signal().
After thread startup:
Each thread enters the print method and executes lock() to acquire the lock. Because ReentrantLock is a mutual exclusion lock, only one thread can hold the lock at any given moment.
Assuming thread A acquires the lock first, it calls a.await(), releasing the lock and entering a waiting state (waiting for the signal of Condition a).
The other threads (B and C) attempt lock(), but the lock is occupied, so they block on lock().
Main thread wakes up thread A:
After try { Thread.sleep(1000); }, the main thread executes awaitSignal.lock(), acquiring the lock.
It calls a.signal(), awakening thread A which is waiting on Condition a.
The main thread executes unlock(), releasing the lock.
After thread A is awakened:
Thread A returns from a.await(), but it needs to reacquire the lock to continue execution.
Because the main thread has already released the lock (unlock()), thread A successfully reacquires the lock.
Thread A prints “a”, then calls b.signal() to wake up thread B.
Thread A executes unlock(), releasing the lock.
After thread B is awakened:
Thread B has been waiting on b.await(), and is awakened after receiving b.signal().
Thread B attempts to reacquire the lock. Since thread A has released the lock, thread B succeeds in acquiring the lock.
Thread B prints “b”, calls c.signal() to awaken thread C, and then releases the lock.

Summary

ReentrantLock can substitute for synchronized to perform synchronization operations.

Acquiring a lock with ReentrantLock is safer.

One must first acquire the lock before entering a try {...} code block, and finally use a finally block to guarantee the lock’s release.

You can use tryLock() to attempt acquiring a lock.

Thread Pool

(The explanations typically found regarding thread pools are rather vague.)

Although Java natively provides multithreading support and starting a new thread is very convenient, creating a thread inherently demands operating system resources (such as thread resources, stack space, etc.). The frequent creation and destruction of massive amounts of threads consume a tremendous amount of time.

What if we could reuse a set of threads:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


┌─────┐ execute ┌──────────────────┐
│Task1│─────────▶│ThreadPool │
├─────┤ │┌───────┐┌───────┐│
│Task2│ ││Thread1││Thread2││
├─────┤ │└───────┘└───────┘│
│Task3│ │┌───────┐┌───────┐│
├─────┤ ││Thread3││Thread4││
│Task4│ │└───────┘└───────┘│
├─────┤ └──────────────────┘
│Task5│
├─────┤
│Task6│
└─────┘
 ...

Then we can have a group of threads execute many small tasks, instead of creating a new thread for each task. This mechanism that accepts large numbers of small tasks and distributes them for processing is called a thread pool.

Simply put, a thread pool internally maintains a set of threads. When there are no tasks, these threads are in a waiting state. When a new task arrives, an idle thread is assigned to execute it. If all threads are busy, the new task is either placed in a queue to wait, or a new thread is created to handle it.

The Java standard library provides the ExecutorService interface representing thread pools, whose typical usage goes as follows:

1
2
3
4
5
6
7
8


// Create a fixed-size thread pool:
ExecutorService executor = Executors.newFixedThreadPool(3);
// Submit tasks:
executor.submit(task1);
executor.submit(task2);
executor.submit(task3);
executor.submit(task4);
executor.submit(task5);

Since ExecutorService is just an interface, the Java standard library provides several common implementations:

FixedThreadPool: A thread pool with a fixed number of threads;
CachedThreadPool: A thread pool that dynamically adjusts its thread count based on the number of tasks;
SingleThreadExecutor: A thread pool that uses only a single thread for execution.

The methods to create these thread pools are all encapsulated in the Executors class. Let’s use FixedThreadPool as an example to see how a thread pool executes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


// thread-pool
import java.util.concurrent.*;

public class Main {
 public static void main(String[] args) {
 // Create a fixed-size thread pool:
 ExecutorService es = Executors.newFixedThreadPool(4);
 for (int i = 0; i < 6; i++) {
 es.submit(new Task("" + i));
 }
 // Shut down the thread pool:
 es.shutdown();
 }
}

class Task implements Runnable {
 private final String name;

 public Task(String name) {
 this.name = name;
 }

 @Override
 public void run() {
 System.out.println("start task " + name);
 try {
 Thread.sleep(1000);
 } catch (InterruptedException e) {
 }
 System.out.println("end task " + name);
 }
}

Looking at the execution results, when 6 tasks are submitted at once, only the first 4 tasks execute simultaneously because the thread pool has a fixed size of 4 threads. The remaining two tasks execute only after some threads become idle.

Thread pools must be shut down when the program terminates. When utilizing the shutdown() method to close a thread pool, it will wait for currently executing tasks to conclude prior to closing. shutdownNow() immediately halts operating tasks, whereas awaitTermination() will delay for a specified period for the thread pool to close sequentially.

If we switch to a CachedThreadPool, since this thread pool implementation dynamically adjusts its size based on the number of tasks, all 6 tasks can execute simultaneously.

What if we wish to confine the thread pool’s size to dynamically adjust between 4 and 10? We inspect the source code of the Executors.newCachedThreadPool() method:

1
2
3
4
5
6


public static ExecutorService newCachedThreadPool() {
 return new ThreadPoolExecutor(
 0, Integer.MAX_VALUE,
 60L, TimeUnit.SECONDS,
 new SynchronousQueue<Runnable>());
}

Thus, to construct a thread pool with a specified dynamic boundary range, we can draft it as:

1
2
3
4
5
6


int min = 4;
int max = 10;
ExecutorService es = new ThreadPoolExecutor(
 min, max,
 60L, TimeUnit.SECONDS,
 new SynchronousQueue<Runnable>());

ScheduledThreadPool

There is another type of task that needs to be executed periodically, for example, refreshing stock prices every second. Such tasks that are fixed in nature and need to run repeatedly can use ScheduledThreadPool. Tasks placed in a ScheduledThreadPool can be executed on a recurring schedule.

Creating a ScheduledThreadPool is still done through the Executors class:

1

ScheduledExecutorService ses = Executors.newScheduledThreadPool(4);

We can submit a one-time task that will be executed once after a specified delay:

1
2


// Execute a one-time task after 1 second:
ses.schedule(new Task("one-time"), 1, TimeUnit.SECONDS);

If a task proceeds on a fixed 3-second routine consistently, we frame it as:

1
2


// Begin a periodic task after 2 seconds, execute every 3 seconds:
ses.scheduleAtFixedRate(new Task("fixed-rate"), 2, 3, TimeUnit.SECONDS);

If tasks execute consecutively spaced with fixed 3-second buffering intervals universally, we implement it as:

1
2


// Begin a periodic task after 2 seconds, execute matching 3-second buffer intervals sequentially:
ses.scheduleWithFixedDelay(new Task("fixed-delay"), 2, 3, TimeUnit.SECONDS);

Note the difference between FixedRate and FixedDelay:

FixedRate means that the task is always triggered at a fixed time interval, regardless of how long the task takes to execute:

1
2
3


│░░░░ │░░░░░░ │░░░ │░░░░░ │░░░ 
├───────┼───────┼───────┼───────┼────▶
│◀─────▶│◀─────▶│◀─────▶│◀─────▶│

FixedDelay, on the other hand, means that after the previous task finishes executing, it waits for a fixed time interval before executing the next task:

1
2
3


│░░░│ │░░░░░│ │░░│ │░
└───┼───────┼─────┼───────┼──┼───────┼──▶
 │◀─────▶│ │◀─────▶│ │◀─────▶│

Therefore, when using a ScheduledThreadPool, we must choose whether to execute a task once, at a fixed rate (FixedRate), or with a fixed delay (FixedDelay), depending on our requirements.

You can also consider the following questions:

In FixedRate mode, assuming a task is triggered every second, if a particular execution takes longer than 1 second, will the subsequent tasks execute concurrently?
If a task throws an exception, will the subsequent tasks continue to execute?

The Java Standard Library also provides the java.util.Timer class, which can execute tasks periodically. However, a single Timer is backed by a single Thread. Because of this, one Timer can only execute one task periodically; to run multiple scheduled tasks, you must start multiple Timer instances. In contrast, a single ScheduledThreadPool can schedule multiple periodic tasks. Therefore, we can completely replace the legacy Timer class with ScheduledThreadPool.

Deploying a Minecraft Server on Linux (CentOS)

Wed, 19 Mar 2025 18:48:59 +0800

I recently set up a physical server and wanted to use it to host a Minecraft server. After staying up late researching, I succeeded and am sharing this guide in hopes that it helps everyone, especially Linux users without a public IP environment (like me).

Reference Websites

Installing and using Red Hat build of OpenJDK 21 on RHEL | Red Hat Product Documentation

SakuraFrp Launcher Installation / Usage Guide | SakuraFrp Documentation

CentOS | Docker Docs

Linux Terminal Server Hosting Tutorial ★ No Panel ★ Minecraft_bilibili

From the Bilibili uploader 翱翔大使, which is the main source of ideas for this whole article.

Java Configuration

Running Minecraft requires the corresponding version of the Java environment; here I installed OpenJDK 21.

1
2


sudo yum install java-21-openjdk
java -version // verify if successfully installed

If the server has multiple Java versions, you can use alternatives to switch versions.

1

alternatives --config java

As shown below, we enter 2 and press Enter to switch to the required version.

Game Deployment

First, download the Minecraft server software from the following URL. Here I downloaded Banner (1.20.1), which supports Fabric.

[MohistMC](MohistMC - Home)

Once downloaded, you will get a file like banner-1.20.1-800-server.jar. Next, open your SSH client to operate on the server:

1
2
3


cd /home/username // switch to personal directory or desired installation location
mkdir Minecraft // create a folder for the game
cd Minecraft

Using the SFTP feature in your SSH client (or any other file transfer method), copy the game file banner-1.20.1-800-server.jar you just downloaded into the newly created /home/username/Minecraft folder.

Next, let’s write a startup script for the server.

1

nano start.sh

Fill in the following content, but note the purpose of each parameter:

-Xmx is the maximum allocated memory, -Xms is the minimum allocated memory. I have 32GB of memory and allocated 6GB to the game (feel free to allocate more).
banner-1.20.1-800-server.jar is the name of the game file you just downloaded.

1
2


java -Xmx6144M -Xms6144m -jar banner-1.20.1-800-server.jar
stty echo

Press Ctrl + O to write, Enter to confirm, and Ctrl + X to exit.

Next, grant execution permissions to start.sh to avoid permission denied issues.

1

chmod 777 start.sh

Then install screen. Simply put, screen is a tool that helps users create independent sessions that can be resumed at any time.

1

yum install screen

screen has the following common commands:

1
2
3


screen -S [name] // create a new screen named "name"
screen -ls // list names and ports of all running screens
screen -r [port] // attach to the screen with the specified port

Next, create a new screen to run the script.

1

screen -S Minecraft

In the newly appeared session, run start.sh.

1

./start.sh

Then everything should go smoothly. I didn’t encounter any errors here. Finally, it comes to ...EULA... asking us to agree to the EULA. Enter true and press Enter. After a short wait, the game server will run successfully on port 25565.

To leave this Minecraft screen, just press Ctrl+A+D.

For game rule changes (like ‘whether cracked players are allowed to join’), you need to modify the contents of server.properties.

Regarding connection: If it’s a cloud server, map the port in the administration panel, then connect to domain:port in your Minecraft client.

However, for a physical server like mine, or a Linux device, or a personal PC without a public IP, we need to proceed with Intranet Penetration.

Intranet Penetration

Here I am using SakuraFrp, which is well-known and reliable in the Minecraft community, for intranet penetration. Other tools are mostly similar.

Docker

SakuraFrp on Linux runs on Docker, so let’s deploy Docker first. The operations entirely follow the official documentation.

1
2


sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

I encountered very slow installation speeds and download failures here. Running the command again solved the problem.

1

sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

After the installation is complete:

1

sudo systemctl enable --now docker

1

sudo docker run hello-world

The run hello-world test above is very likely to fail. Let’s solve this issue by referring to the following two articles:

[Complete Solution] Failed to run hello-world image after Docker installation: Unable to find image ‘hello-world:latest’ locally - CSDN Blog

Failed or Timeout Running hello-world Image in Docker - Paul7777 - cnblogs

Combining the two above will eventually solve the problem. First, let’s configure the daemon file.

1

nano /etc/docker/daemon.json

Copy the following content into it:

At the time of my testing (2025/3/19), the following mirrors are still working.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


{
 "registry-mirrors": [
 "https://h59pkpv6.mirror.aliyuncs.com",
 "https://registry.docker-cn.com",
 "https://docker.mirrors.ustc.edu.cn",
 "https://hub-mirror.c.163.com",
 "https://mirror.baidubce.com",
 "https://do.nark.eu.org",
 "https://dc.j8.work",
 "https://docker.m.daocloud.io",
 "https://dockerproxy.com",
 "https://docker.nju.edu.cn"
]
}

Save + Exit. Next, restart docker, and execute the test once more.

1
2
3


sudo systemctl daemon-reload
sudo systemctl restart docker
docker run hello-world

I successfully installed docker up to this point. If the test still fails, please check the content of daemon.json to see if there are missing or extra commas and brackets.

SakuraFrp

For deploying SakuraFrp on Linux, the official documentation provides a detailed solution.

First, run the following command as an administrator in the terminal:

1

sudo bash -c ". <(curl -sSL https://doc.natfrp.com/launcher.sh)"

After installation, it should automatically output logs and prompt you to fill in the access token. This (or the subsequent operations) can be found in the management panel on the SakuraFrp official website.

After logging in, you will be able to see its log files. Below are the common operations to start it and view logs:

1
2


docker start natfrp-service
docker logs natfrp-service

As shown below, we now need to physically operate the server.

Open a browser (Linux usually comes with Firefox) and access the URL after “Usage” to open the WebUI.

Then you will see there is nothing under “Tunnels”, only a plus sign. At this time, we enter the SakuraFrp management panel, find Tunnel List under Services, and create two new tunnels as shown below:

The first one with port 7102 is the WebUI for SakuraFrp on the server, intended for remote management.

The second one with port 25565 is for the Minecraft server.

Return to the WebUI interface and refresh, you will see the two tunnels just created. Double-click them respectively, and then go back to the terminal log interface.

The links in red characters as shown in the image are the remote access links for the WebUI and Minecraft. Just copy the Minecraft one into the game, and you can connect to it.

Conclusion

We are done! (A screenshot of my server’s spawn point >w<)

Redis

Tue, 18 Mar 2025 15:41:00 +0800

Redis (Remote Dictionary Server) is a high-performance, open-source, in-memory data structure store, used as a database, cache, and message broker.

References

知乎超强、超详细Redis入门教程

CSDN【Redis二三事】一套超详细的Redis学习教程（步骤图片+实操）—第一集

Detailed, with real business scenario examples.

Data Structures

Redis includes five major data types: string (string), list (list), hash (hash), set (set), and sorted set (zset).

string

The most basic data type in Redis. Each key corresponds to a value, which can be text, numbers, or binary data, with a maximum storage of 512MB. It supports operations like string concatenation, substring, increment, and decrement, and is suitable for scenarios such as caching data, counters (e.g., page view statistics), and distributed locks.

Basic Operations

1
2
3


set key value
get key
del key 

Add/Modify Multiple Data

1

mset key1 value1 key2 value2...

Get Multiple Data

1

mget key1 key2...

Get Character Count of Data

1
2
3
4
5


strlen key
// For example
set name1 nosql
strlen name1
// Output: 5

Append Information

1
2
3
4
5


append key value
// For example
append name1 name
get name1
/* Output: nosqlname*/

Multi-data vs Single-data Operations

Executing $n$ commands individually requires 1 send + 1 process + 1 return $n$ times
Executing $n$ commands via a multi-data command requires 1 send + $n$ processes + 1 return
When the data volume is large, the time consumed by multi-data commands is much less than individual commands

Advanced Operations

Increment/Decrement Numeric Data by a Specified Range

1
2


incrby key increment
decrby key increment

Numeric Operations on String Types

1
2
3
4
5
6
7
8


127.0.0.1:6379> set mynum "2"
OK
127.0.0.1:6379> get mynum
"2"
127.0.0.1:6379> incr mynum
(integer) 3
127.0.0.1:6379> get mynum
"3"

When encountering numeric operations, Redis will automatically convert the string type into a number.

Considerations for String Type Numeric Operations

Differences between execution status feedback and normal data manipulation feedback
- Indicates whether the operation results were successful
  - (integer) 0 -> false Failed
  - (integer) 1 -> true Succeeded
- Indicates the resulting value
  - (integer) 3 -> 3 3 items
  - (integer) 1 -> 1 1 item
Data not found
- (nil) is equivalent to null
Maximum data storage capacity
- 512MB
Maximum range for numeric calculations (maximum value of long in java)
- 9223372036854775807

hash

Similar to a small key-value store, suitable for storing structured data such as user information (ID, name, email, etc.). Compared to the String type, it saves more memory because multiple fields share the same key. You can operate on fields individually to avoid reading and modifying the entire object, making it suitable for storing objects, session information, etc.

Basic Operations

1
2
3
4


hset key field value // Add/Modify data
hget key field // Get data
hgetall key
hdel key field1 [field2]

Add/Modify Multiple Data

1

hmset key field1 value1 field2 value2

Get Multiple Data

1

hmget key field1 field2...

Get the Number of Fields in a Hash Table

1

hlen key

Check if a Specified Field Exists in a Hash Table

1

hexists key field

Advanced Operations

Get All Field Names or Field Values in a Hash Table

1
2


hkeys key
hvals key

Increment the Numeric Value of a Specified Field by a Certain Range

1

hincrby key field increment

Comprehensive Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


// Create hash and assign values
127.0.0.1:6379> HMSET user:001 username antirez password P1pp0 age 34 
OK
// List the contents of the hash
127.0.0.1:6379> HGETALL user:001 
1) "username"
2) "antirez"
3) "password"
4) "P1pp0"
5) "age"
6) "34"
// Change a specific value in the hash
127.0.0.1:6379> HSET user:001 password 12345 
(integer) 0
// List the contents of the hash again
127.0.0.1:6379> HGETALL user:001 
1) "username"
2) "antirez"
3) "password"
4) "12345"
5) "age"
6) "34"

Considerations for Hash Type Data Operations

1. The value under the hash type can only store strings, and no other data types are allowed, so there is no nesting. If no data is found, the corresponding value is (nil).
1. Each hash can store 2^23-1 key-value pairs.
1. The hash type is very close to the data storage format of objects, and object properties can be flexibly added. However, its initial design was not meant for storing large numbers of objects. Remember not to abuse it, and never use a hash as an object list.
1. The hgetall operation can retrieve all properties. If there are too many fields inside, traversing the whole data will be very inefficient and may become a bottleneck for data access.

list

Based on a doubly linked list, it allows fast insertion and deletion of elements from the head (left) or tail (right), and supports reading elements by specifying an index range. It is suitable for implementing message queues, timelines (such as Weibo feeds), task scheduling, and other applications, especially for scenarios that require processing data according to insertion order.

Basic Operations

Add/Modify Data

1
2


lpush key value1 [value2]... // Push from the left
rpush key value1 [value2]... /* Push from the right */

Get Data

About lrange:

lrange is used to get elements within a specified range
-1 represents the last element
List element indexing starts from position 0

1
2
3


lrange key start stop
lindex key index
lien key

Get and Remove Data

1
2


lpop key
rpop key

Remove Specified Data

About lrem:

Parameter count
- count > 0 -> Remove count matching values starting from the head (left)
- count < 0 -> Remove count matching values starting from the tail (right)
- count = 0 -> Remove all matching values (equivalent to removing all elements with this value from the list)
If key does not exist
- Returns 0
Commonly used to remove duplicate elements from a list or clean up data

1

lrem key count value

Advanced Operations

Get and Remove Data Within a Specified Time

About blpop:

Parameter key1 [key2...]
- Multiple list keys can be provided, and Redis will check these lists in order sequentially
Parameter timeout
- timeout > 0 -> If the list is empty, wait up to timeout seconds
- timeout < 0 -> Block indefinitely until data is available
Applicable to scenarios like task queues, producer-consumer models, etc.

1
2


blpop key1 [key2] timeout
brpop key1 [key2] timeout

Example

1
2
3
4
5


RPUSH list1 "a" "b" "c" // List content: ["a", "b", "c"]
BLPOP list1 10 // Pop "a", returns ["list1", "a"]
BLPOP list1 10 // Pop "b", returns ["list1", "b"]
BLPOP list1 10 // Pop "c", returns ["list1", "c"]
BLPOP list1 10 /* List is empty, blocks for up to 10 seconds; if no new elements, returns nil */

Comprehensive Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


// Create a new list called mylist and insert element "1" at the head
127.0.0.1:6379> lpush mylist "1" 
// Returns the current number of elements in mylist
(integer) 1 
// Insert element "2" on the right side of mylist
127.0.0.1:6379> rpush mylist "2" 
(integer) 2
// Insert element "0" on the left side of mylist
127.0.0.1:6379> lpush mylist "0" 
(integer) 3
// List elements in mylist from index 0 to 1
127.0.0.1:6379> lrange mylist 0 1 
1) "0"
2) "1"
// List elements in mylist from index 0 to the last element
127.0.0.1:6379> lrange mylist 0 -1 
1) "0"
2) "1"
3) "2"

Considerations for List Type Data Operations

1. The data stored in a list is of the string type, and the total data capacity is limited to a maximum of 2^32-1 elements.
1. A list has the concept of “indexes,” but manipulating data is usually done in the form of a “queue” (enqueue/dequeue) or a “stack” (push/pop).
1. When retrieving all data, the end index is set to -1.
1. A list allows pagination of data; typically, page 1 information comes from the list, while page 2 and more are loaded in a “database” fashion.

set

It consists of unique, unordered elements, supporting addition, deletion, and search operations with O(1) time complexity, and provides set operations like intersection, union, and difference. Suitable for applications like deduplication, mutual follows in recommendation systems, and tag management. Because duplicate elements are not allowed, it efficiently stores collections of distinct data.

Basic Operations

Add Data

1

sadd key member1 [member2]

Get All Data

1

smembers key

Delete Data

1

srem key member1 [member2]

Get Total Amount of Set Data

1

scard key

Check if Specified Data is in the Set

1

sismember key member

Advanced Operations

Randomly Get a Specified Number of Elements from the Set

1

srandmember key [count]

Randomly Get an Element from the Set and Remove It

1

spop key

Calculate the Intersection, Union, and Difference of Two Sets

1
2
3


sinter key1 [key2]
sunion key1 [key2]
sdiff key1 [key2]

Calculate the Intersection, Union, and Difference of Two Sets and Store in a Specified Set

1
2
3


sinterstore destination key1 [key2]
sunionstore destination key1 [key2]
sdiffstore destination key1 [key2]

Move Specified Data from the Source Set to the Destination Set

1

smove source destination member

Comprehensive Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


// Add a new element "one" to the set myset
127.0.0.1:6379> sadd myset "one" 
(integer) 1
127.0.0.1:6379> sadd myset "two"
(integer) 1
// List all elements in the set myset
127.0.0.1:6379> smembers myset 
1) "one"
2) "two"
// Check if element "one" is in the set myset, returning 1 indicates existence
127.0.0.1:6379> sismember myset "one" 
(integer) 1
// Check if element "three" is in the set myset, returning 0 indicates non-existence
127.0.0.1:6379> sismember myset "three" 
(integer) 0
// Create a new set yourset
127.0.0.1:6379> sadd yourset "1" 
(integer) 1
127.0.0.1:6379> sadd yourset "2"
(integer) 1
127.0.0.1:6379> smembers yourset
1) "1"
2) "2"
// Take the union of the two sets
127.0.0.1:6379> sunion myset yourset 
1) "1"
2) "one"
3) "2"
4) "two"

Considerations for Set Type Data Operations

The set type does not allow duplicate data. If the added data already exists in the set, only one copy will be kept.
Although set has the same storage structure as hash, it cannot utilize the space for storing values in the hash.

zset

Builds upon Set by adding a score to each element and ordering them by this score, supporting ranged queries, score-based ranking, and other operations. Suitable for leaderboards (like game scores), priority queues (like scheduled tasks), and time-sorted data storage (like article reading rankings), where sorting by weight is needed.

Basic Operations

Add Data

1

zadd key score1 member1 [score2 member2]

Get All Data

1
2


zrange key start stop [WITHSCORES] // Display in ascending order
zrevrange key start stop [WITHSCORES] // Display in descending order

Delete Data

1

zrem key member [member...]

Get Data by Condition

1
2


zrangebyscore key min max [Withscores][limit]
zrevrangebyscore key max min [withscores]

Delete Data by Condition

1
2


zremrangebyrank key start stop // Delete by index
zremrangebyscore key min max 	// Delete by range

Comprehensive Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


// Add a new sorted set myzset, insert the element baidu.com, and assign it score 1:
127.0.0.1:6379> zadd myzset 1 baidu.com 
(integer) 1
// Add the element 360.com to myzset, assigning it score 3
127.0.0.1:6379> zadd myzset 3 360.com 
(integer) 1
// Add the element google.com to myzset, assigning it score 2
127.0.0.1:6379> zadd myzset 2 google.com 
(integer) 1
// List all elements of myzset along with their scores, showing that myzset is already ordered.
127.0.0.1:6379> zrange myzset 0 -1 with scores 
1) "baidu.com"
2) "1"
3) "google.com"
4) "2"
5) "360.com"
6) "3"
// List only the elements of myzset
127.0.0.1:6379> zrange myzset 0 -1 
1) "baidu.com"
2) "google.com"
3) "360.com"

Advanced Operations

Get Total Amount of Set Data

1
2


zcard key
zcount key min max

Set Intersection and Union Operations

1
2


zinterstore destination numkeys key [key …]
zunionstore destination numkeys key [key …]

Considerations for sorted_set Type Data Operations

The data storage space for the score is 64-bit; for an integer, the range is -9007199254740992 to 9007199254740992. The data saved in the score can also be a double-precision floating-point number. Based on the characteristics of double-precision floating-point numbers, precision might be lost, so it should be used with caution. The underlying storage of sorted_set is still based on the set structure. Therefore, data cannot be duplicated. If identical data is added repeatedly, the score value will simply be overwritten repeatedly, keeping the result of the last modification.

MySQL

Mon, 17 Mar 2025 20:06:34 +0800

References

自学SQL网(教程视频练习全套)

Learn the knowledge and immediately have exercises to do.

MySQL 教程 | 菜鸟教程

MySQL总结_sq连表-CSDN博客

主键 - SQL教程 - 廖雪峰的官方网站

Best viewing experience tutorial.

Relational Model

Quoted from 关系模型 - SQL教程 - 廖雪峰的官方网站

Primary Key

In a relational database, each row of data in a table is called a record. A record is composed of multiple fields. For example, two records in the students table:

id	class id	name	gender	score
1	1	Xiaoming	M	90
2	1	Xiaohong	F	95

For a relational table, there is a very important constraint: any two records cannot be duplicated. Non-duplication doesn’t mean two records are not entirely identical, but rather that different records can be uniquely distinguished by a certain field, and this field is called the primary key.

For example, assuming we use the name field as the primary key, then we can uniquely identify a record through the name Xiaoming or Xiaohong. However, with this setup, we couldn’t store students with the same name, because inserting two records with the same primary key is not allowed.

The most crucial requirement for a primary key is: once a record is inserted into the table, it’s best not to modify the primary key anymore, because the primary key is used to uniquely locate a record. Modifying the primary key will cause a series of cascading effects.

Because the role of the primary key is so important, how to select a primary key will have a significant impact on business development. If we use a student’s ID number as the primary key, it seems to uniquely locate a record. However, an ID number is also a business scenario. If the ID number’s length increases or needs to be changed, and as a primary key, it has to be modified, it will have a severe impact on the business.

Therefore, a basic principle for selecting a primary key is: do not use any business-related fields as the primary key.

Thus, fields that seem unique like ID numbers, mobile phone numbers, and email addresses, should not be used as primary keys.

The best primary key is a field completely unrelated to the business, and we generally name this field id. Common types suitable for the id field are:

Auto-incrementing integer type: The database will automatically assign an auto-incrementing integer to each record upon insertion, so we don’t have to worry at all about primary key duplication, nor do we need to pre-generate the primary key ourselves;
Globally Unique Identifier (GUID) type: Also known as UUID, using a globally unique string as a primary key, such as 8f55d96b-8acc-4636-8cb8-76bf8abc2f57. The GUID algorithm ensures that the strings generated by any computer at any time are different through network card MAC addresses, timestamps, and random numbers. Most programming languages have built-in GUID algorithms, allowing you to pre-calculate the primary key.

For most applications, an auto-incrementing primary key is usually sufficient. The primary key we defined in the students table is also of BIGINT NOT NULL AUTO_INCREMENT type.

If an INT auto-increment type is used, an error will occur when the number of records in a table exceeds 2,147,483,647 (about 2.1 billion) as it reaches the upper limit. Using the BIGINT auto-increment type allows for a maximum of about 9.22 quintillion records.

Summary

The primary key is the unique identifier for a record in a relational table. Selecting a primary key is very important: a primary key should not have any business meaning, and should instead use a BIGINT auto-increment or GUID type. A primary key should also not allow NULL.

Multiple columns can be used as a composite primary key, but composite primary keys are not commonly used.

Foreign Key

When we uniquely identify records using a primary key, we can determine any student’s record in the students table:

id	name	other columns…
1	Xiaoming	…
2	Xiaohong	…

We can also determine any class record in the classes table:

id	name	other columns…
1	Class 1	…
2	Class 2	…

But how do we determine which class a record in the students table belongs to, for example, Xiaoming with id=1?

Since one class can have multiple students, in the relational model, the relationship between these two tables can be called “one-to-many”, meaning one record in the classes table can correspond to multiple records in the students table.

To express this one-to-many relationship, we need to add a class_id column to the students table, letting its value correspond to a specific record in the classes table:

id	class_id	name	other columns…
1	1	Xiaoming	…
2	1	Xiaohong	…
5	2	Xiaobai	…

This way, we can directly locate which record in the classes table a students table record should correspond to based on the class_id column.

Xiaoming’s class_id is 1, therefore, the corresponding record in the classes table is Class 1 with id=1;
Xiaohong’s class_id is 1, therefore, the corresponding record in the classes table is Class 1 with id=1;
Xiaobai’s class_id is 2, therefore, the corresponding record in the classes table is Class 2 with id=2.

In the students table, through the class_id field, data can be associated with another table, and such a column is called a foreign key.

A foreign key is not implemented through the column name, but by defining a foreign key constraint:

1
2
3
4


ALTER TABLE students
ADD CONSTRAINT fk_class_id
FOREIGN KEY (class_id)
REFERENCES classes (id);

Here, the foreign key constraint name fk_class_id can be anything, FOREIGN KEY (class_id) specifies class_id as the foreign key, and REFERENCES classes (id) specifies that this foreign key will be linked to the id column of the classes table (i.e., the primary key of the classes table).

By defining a foreign key constraint, a relational database can guarantee that invalid data cannot be inserted. That is, if a record with id=99 doesn’t exist in the classes table, the students table cannot insert a record with class_id=99.

Since foreign key constraints reduce database performance, most Internet applications, in pursuit of speed, do not set foreign key constraints and solely rely on the application itself to ensure logical correctness. In this case, class_id is just an ordinary column, it simply acts as a foreign key.

To delete a foreign key constraint, it is also achieved via ALTER TABLE:

1
2


ALTER TABLE students
DROP FOREIGN KEY fk_class_id;

Note: Deleting a foreign key constraint does not delete the foreign key column itself. Deleting a column is achieved via DROP COLUMN ....

Many-to-Many

By associating a foreign key in one table with another table, we can define a one-to-many relationship. Sometimes, we also need to define a “many-to-many” relationship. For example, one teacher can correspond to multiple classes, and one class can also correspond to multiple teachers. Thus, there is a many-to-many relationship between the class table and the teacher table.

A many-to-many relationship is actually implemented through two one-to-many relationships. That is, by using an intermediate table to connect two one-to-many relationships, a many-to-many relationship is formed:

teachers table:

id	name
1	Mr. Zhang
2	Mr. Wang
3	Mr. Li
4	Mr. Zhao

classes table:

id	name
1	Class 1
2	Class 2

Intermediate table teacher_class connecting two one-to-many relationships:

id	teacher_id	class_id
1	1	1
2	1	2
3	2	1
4	2	2
5	3	1
6	4	2

Through the intermediate table teacher_class, we can know the relationship from teachers to classes:

Mr. Zhang with id=1 corresponds to Class 1 and Class 2 with id=1,2;
Mr. Wang with id=2 corresponds to Class 1 and Class 2 with id=1,2;
Mr. Li with id=3 corresponds to Class 1 with id=1;
Mr. Zhao with id=4 corresponds to Class 2 with id=2.

Similarly, we can know the relationship from classes to teachers:

Class 1 with id=1 corresponds to Mr. Zhang, Mr. Wang, and Mr. Li with id=1,2,3;
Class 2 with id=2 corresponds to Mr. Zhang, Mr. Wang, and Mr. Zhao with id=1,2,4;

Therefore, through the intermediate table, we’ve defined a “many-to-many” relationship.

One-to-One

A one-to-one relationship means that a record in one table corresponds to a single, unique record in another table.

For instance, every student in the students table can have their own contact information. If we store the contact details in another table contacts, we can obtain a “one-to-one” relationship:

id	student_id	mobile
1	1	135xxxx6300
2	2	138xxxx2209
3	5	139xxxx8086

Some attentive readers might ask, since it’s a one-to-one relationship, why not just add a mobile column to the students table so they can be merged into one?

If the business logic allows, combining the two tables into one is entirely possible. However, sometimes if a student doesn’t have a mobile number, a corresponding record wouldn’t exist in the contacts table. In fact, a one-to-one relationship, strictly speaking, is the contacts table having a one-to-one correspondence with the students table.

Furthermore, some applications split a large table into two one-to-one tables to separate frequently read fields from infrequently read ones for better performance. For example, splitting a large user table into a basic user information table user_info and a detailed user information table user_profiles. Most of the time, only the user_info table needs to be queried without querying user_profiles, which improves the query speed.

Summary

Relational databases can implement one-to-many, many-to-many, and one-to-one relationships using foreign keys. Foreign keys can either be constrained by the database or set without constraints, relying solely on the application’s logic to guarantee integrity.

Index

In a relational database, if there are tens of thousands or even hundreds of millions of records, you need to use indexes to achieve very fast query speeds.

An index is a strictly pre-sorted data structure in a relational database for the values of one or multiple columns. By utilizing indexes, the database system doesn’t have to scan the entire table, but directly pinpoints the records that meet the criteria, greatly speeding up queries.

For example, for the students table:

id	class_id	name	gender	score
1	1	Xiaoming	M	90
2	1	Xiaohong	F	95
3	1	Xiaojun	M	88

If you frequently query based on the score column, you can create an index on the score column:

1
2


ALTER TABLE students
ADD INDEX idx_score (score);

Using ADD INDEX idx_score (score) creates an index named idx_score that utilizes the score column. The index name is arbitrary, and if the index comprises multiple columns, they can be written sequentially in parentheses, for example:

1
2


ALTER TABLE students
ADD INDEX idx_name_score (name, score);

The efficiency of an index depends on whether the values of the indexed column are dispersed—that is, the more distinct the column’s values are, the higher the index efficiency. Conversely, if a column’s records contain a large number of identical values, like the gender column where roughly half the values are M and the other half are F, creating an index for that column makes no sense.

You can create multiple indexes for a single table. The advantage of indexes is improved query efficiency, while the disadvantage is that when inserting, updating, and deleting records, the index needs to be modified simultaneously. Therefore, the more indexes there are, the slower the operations to insert, update, and delete records become.

For primary keys, relational databases will automatically create a primary key index for them. Using a primary key index is the most efficient because the primary key guarantees absolute uniqueness.

Unique Index

When designing relational data tables, columns that appear unique, such as ID numbers and email addresses, should not be modeled as primary keys due to having business significance.

However, based on business requirements, these columns still have a uniqueness constraint: meaning two records cannot store the exact same ID number. At this point, we can add a unique index to this column. For example, assuming the name in the students table cannot be duplicated:

1
2


ALTER TABLE students
ADD UNIQUE INDEX uni_name (name);

Through the UNIQUE keyword, we have added a unique index.

You can also add a unique constraint to a certain column without creating a unique index:

1
2


ALTER TABLE students
ADD CONSTRAINT uni_name UNIQUE (name);

In this scenario, the name column doesn’t have an index, but still maintains a uniqueness guarantee.

Regardless of whether an index is created or not, using a relational database will make no difference to the user and the application. This implies that when we query the database, if a matching index is available, the database system will automatically utilize the index to boost query efficiency. If no index exists, the query will still execute normally, but at a slower speed. Hence, indexes can be progressively optimized during database usage.

Summary

Creating indexes for database tables can accelerate query speeds;

Creating unique indexes acts to guarantee the uniqueness of the values in a specific column;

Database indexes are transparent to both users and applications.

SELECT Query

1
2
3
4
5
6
7
8


SELECT column, another_column, …
FROM mytable
WHERE condition
 AND/OR another_condition
 AND/OR …;
 
SELECT * FROM movies 
WHERE year>=2010 AND length_minutes<120;

Filtering Numeric Attribute Columns

Keyword		Example
=, !=, < <=, >, ≥		col_name != 4
BETWEEN … AND …	Between two numbers	col_name BETWEEN 1.5 AND 10.5
NOT BETWEEN … AND …		col_name NOT BETWEEN 1 AND 10
IN (…)	In a list	col_name IN (2, 4, 6)
NOT IN (…)		col_name NOT IN (1, 3, 5)

Filtering String Attribute Columns


=	Exactly equals
!= or <>	Does not equal
LIKE	Equivalent to = without wildcards
NOT LIKE	Equivalent to != without wildcards
%	Wildcard	col_name LIKE “%AT%”
_(Underscore)		col_name LIKE “AN_”

1
2
3
4
5
6
7


/* Wildcards */
col_name LIKE "%AT%";
/* "AT", "AT*...", "...*AT", "...*AT*..." all satisfy the condition
 There can be arbitrary characters before and after "AT" */
col_name LIKE "AN_";
/* "AND" is okay, "AN", "ANDD" are not
 Similar to '%', but only represents a single character */

Filtering / Sorting

1
2
3
4


/* Use the DISTINCT keyword to specify that a certain attribute column or columns return uniquely */
SELECT DISTINCT column, another_column, …
FROM mytable
WHERE condition(s);

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


/* Sort the results based on one or more attribute columns */
SELECT column, another_column, …
FROM mytable
WHERE condition(s)
/* ASC for ascending or DESC for descending */
ORDER BY column ASC/DESC
/* LIMIT specifies how many rows of results to return
 OFFSET specifies from which row to start returning */
LIMIT num_limit OFFSET num_offset;
/* Regarding OFFSET, if you want to output the Nth row (and onwards)
 the parameter for OFFSET must be N-1 */

Example Problem

SELECT Review Problems

Using Expressions in Queries

Actually, AS is not only used for assigning aliases to expressions; standard attribute columns and even tables can be assigned an alias, making the SQL easier to grasp.

1
2
3
4
5


-- Examples of aliasing attribute columns and tables
SELECT column AS better_column_name, …
FROM a_long_widgets_table_name AS mywidgets
INNER JOIN widget_sales
 ON mywidgets.id = widget_sales.widget_id;

1
2
3
4
5


-- Example containing an expression
SELECT particle_speed / 2.0 AS half_particle_speed -- Divided the results by 2
FROM physics_data
WHERE ABS(particle_position) * 10.0 >500
 -- (The condition requires the absolute value of this attribute multiplied by 10 to be greater than 500);

Performing Statistics in Queries

1
2
3


SELECT AGG_FUNC(column_or_expression) AS aggregate_description, …
FROM mytable
WHERE constraint_expression;

Common statistical functions:

Function	Description
COUNT(*) COUNT(column)	Counting! COUNT(*) counts the number of data rows, COUNT(column) counts the number of non-NULL rows in the column
MIN(column)	Finds the row with the smallest column value
MAX(column)	Finds the row with the largest column value
AVG(column)	Takes the average of the column for all rows
SUM(column)	Sums the column for all rows

Grouped Statistics

The GROUP BY data grouping syntax can group data by a specific col_name. For example, GROUP BY Year means grouping the data by year, placing data from the same year into the same group. If a statistical function is combined with GROUP BY, then the statistical result is constrained to the data within each group. The number of data rows resulting from a GROUP BY grouping is exactly the number of groups. For instance, with GROUP BY Year, however many years exist in the overall data, that same number of data rows will be returned, regardless of whether a statistical function is applied.

1
2
3
4
5


-- Statistics using groupings
SELECT AGG_FUNC(column_or_expression) AS aggregate_description, …
FROM mytable
WHERE constraint_expression
GROUP BY column;

In the GROUP BY grouping syntax, we know that the database first filters the data with WHERE, and then groups the results. What if we want to filter out a few more rows from the already grouped data? A less commonly utilized syntax, the HAVING syntax, will be employed to resolve this problem; it allows further SELECT filtering on the post-grouping data.

The HAVING syntax is identical to WHERE, except the result set it operates on is different. In the case of the small datasets in our examples, HAVING may not seem very useful, but when your data volume hits the thousands or millions with numerous attributes, it can be of immense help.

JOIN Connections

Database Normal Forms

Database normal forms are standardizations for data table design. Under these normal form guidelines, the duplicate data stored by each schema is reduced to a minimum (assisting in maintaining data coherence). Meanwhile, under database normalization, tables no longer possess rigid data decoupling, permitting independent scaling (i.e., for example, the growth of car engines and cars is completely decoupled).

1
2
3
4
5
6
7
8


SELECT column, another_table_column, …
FROM mytable -- The primary table
INNER JOIN another_table -- The table to be joined
 ON mytable.id = another_table.id 
 -- Imagine the primary key join mentioned earlier, two identical ones are merged into 1 row
WHERE condition(s)
ORDER BY column, … ASC/DESC
LIMIT num_limit OFFSET num_offset;

The relation association described by the ON condition in this example:

`INNER JOIN` (Inner) Connections

First, combine the data from two tables together, any data from either table that fails to find a counterpart via ID will be discarded. At this stage, you may envision the post-join data as an aggregation of two tables, where the remaining SQL clauses will continue their execution upon this combination (imagine it’s exactly the same as the previous single-table operations). Another method for comprehending an INNER JOIN is to think of an INNER JOIN as the intersection of two sets.

`OUTER JOIN` Outer Connections

1
2
3
4
5
6
7
8


-- Perform multi-table queries using LEFT/RIGHT/FULL JOINs
SELECT column, another_column, …
FROM mytable
INNER/LEFT/RIGHT/FULL JOIN another_table
 ON mytable.id = another_table.matching_id
WHERE condition(s)
ORDER BY column, … ASC/DESC
LIMIT num_limit OFFSET num_offset;

When executing a connection on Table A to Table B, a LEFT JOIN retains all elements of A, unaffected by if they match successfully with B, conversely, a RIGHT JOIN retains all elements present inside B. Concluding, a FULL JOIN will simultaneously conserve all row items originating from both A and B regardless of achieving any matches.

Establishing a 1-to-1 linkage connecting two table schemas reserves A’s or B’s intrinsic entries, where if a provided column doesn’t reside within the opposite table, a NULL is to fill up the consequential data.

Query Execution Order

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


-- This is the complete SELECT query
SELECT DISTINCT column, AGG_FUNC(column_or_expression), …
FROM mytable
 JOIN another_table
 ON mytable.column = another_table.column
 WHERE constraint_expression
 GROUP BY column
 HAVING constraint_expression
 ORDER BY column ASC/DESC
 LIMIT count OFFSET COUNT;

1. `FROM` and `JOIN`

FROM or JOIN are the first to execute, pinpointing the overall constraints behind a data range. Should different tables require a JOIN, an intermediate temporary Table may generate, subsequently serving later downstream operations. Fundamentally, this initial block could be summarized as recognizing the genesis data source table (inclusive of any temp tables).

2. `WHERE`

Now that we have ascertained the data schema source, the WHERE statement undertakes data screening on this data origin depending exclusively on specified parameters, ejecting whichever data entries do not correspond to the requested qualifications. All screened col attributes ought to materialize out of the tables identified through FROM. Due to aliases plausibly embodying non-executed formulas, AS aliases cannot be invoked dynamically inside this specific phase.

3. `GROUP BY`

If you implemented GROUP BY for classifying entries, the subsequent GROUP BY groups earlier data alongside computations for tallies, shrinking resultant outcomes aligned closer to the aggregate number of partitions. Therefore, any extra parameters not part of the classified dimensions get wiped.

4. `HAVING`

If you implemented GROUP BY clustering, HAVING conducts additional sifting onto corresponding outcome schemas immediately after finishing earlier groupings. AS aliases conversely stay deactivated for use amidst this procedure phase.

5. `SELECT`

After finalizing findings, SELECT is implemented across columns comprising the solution output to perform simple sifting operations or computational mathematics aimed specifically at establishing the explicit contents outputted.

6. `DISTINCT`

Should elements suffer uninvited repetition, DISTINCT bears the duty to enforce deduplication.

7. `ORDER BY`

Where the generated finding bounds stay solidly solidified, ORDER BY categorizes sequence alignments over the end-product. Since standard evaluation across variables enveloped in SELECT rests fulfilled. Thus here entails dynamic utility via establishing AS aliases.

8. `LIMIT` / `OFFSET`

As the conclusion steps, LIMIT beside OFFSET carve out compartmental slices directly abstracted right after sorted listings.

Conclusion

Not absolutely every SQL statement relies intrinsically around comprehensively capturing all potential terminologies, but nimbly navigating via varied compositional constructs paired securely closely with intuitive SQL theoretical execution foundations ensures superior resolutions handling localized database hurdles exclusively bounded natively on SQL, rather than blindly migrating all problematic dilemmas towards application code programming abstractions.

NULL

If you overlook committing distinct inputs allocated for designated database columns, it prominently projects a NULL. Thus, a recurrent solution allocates structural default values, typically like standardizing numeric fields to 0, combined with calibrating textual dimensions matching "" string dimensions. Nevertheless, where intrinsic characteristics authentically mirror absolute intentional NULL meanings, please pay cautious consideration against recklessly standardizing alternative defaults versus legitimately accepting pure NULL. (For example, whilst establishing cross-record average arithmetic, applying 0 automatically registers evaluations consequently contaminating accurate calculations, although deploying pure NULL exclusively bypasses erroneous quantitative inclusions entirely).

Another situational bracket proving highly restrictive preventing NULL generations involves executing outer-joining multi-schema combinations earlier narrated, considering whenever quantitative data discrepancies present amongst A contrasting B schemas, one absolutely depends directly around NULL to seamlessly patch up dimensions natively missing references. Tackling equivalent predicaments, manipulating logic gates bounded internally with IS NULL identically accompanied by IS NOT NULL expertly determines specific boolean verifications judging if a definitive spatial coordinate explicitly registers fundamentally parallel corresponding precisely toward NULL.

1
2
3
4
5


SELECT column, another_column, …
FROM mytable
WHERE column IS/IS NOT NULL
AND/OR another_condition
AND/OR …;

Modifying Data

Partially quoted from 修改数据 - SQL教程 - 廖雪峰的官方网站

The bedrock operational foundations revolving around relational database constructs categorically parallel CRUD manipulations: Create, Retrieve, Update, Delete. Addressing specific interrogations, we comprehensively expounded exactly describing elaborate dynamic utilities encompassing the SELECT statements effectively.

Transitioning precisely over dimensional expansions, excisions alongside modifications dynamically reflect parallel standard matching respective descriptive SQL queries:

INSERT: Inject original fresh record items;
UPDATE: Overwrite formerly registered content;
DELETE: Terminate and rip historical documented dimensions out.

We will singularly unpack distinct structural application semantics independently driving these tri-fold modifying algorithmic assertions consecutively.

INSERT Insertion

Referenced in MySQL 插入数据 | 菜鸟教程

For illustration, injecting completely novel unrepresented components directly supplementing an internal user structural matrix primarily lists requisite sequential locational destinations dynamically accompanied by parallel structured inputs categorically sequentially encoded straightly following downstream VALUES parameters:

1
2
3
4
5


INSERT INTO table_name (column1, column2, column3, ...)
VALUES (value1, value2, value3, ...);
--
INSERT INTO users (username, email, birthdate, is_active)
VALUES ('test', 'test@runoob.com', '1990-01-01', true);

Whenever attempting holistic whole scale broad injections globally accommodating all columns (which ultimately equates loading literal standalone rows natively), explicit dimensional coordinates remain perfectly dismissible entirely voluntarily:

1
2


INSERT INTO users
VALUES (NULL,'test', 'test@runoob.com', '1990-01-01', true);

Moreover it is similarly trivial establishing multifaceted multi-tier simultaneous bundled row records entirely unified through simply assigning combined aggregated sets grouped intrinsically under VALUES clauses, sequentially mapping individualized items cleanly compartmentalized bound uniquely using encapsulated parenthesis formatted like (...) neatly divided employing systematic commas , effectively:

1
2
3
4
5


INSERT INTO users (username, email, birthdate, is_active)
VALUES
 ('test1', 'test1@runoob.com', '1985-07-10', true),
 ('test2', 'test2@runoob.com', '1988-11-25', false),
 ('test3', 'test3@runoob.com', '1993-05-03', true);

UPDATE Undergoing Updates

MySQL UPDATE 更新 | 菜鸟教程

1
2
3


UPDATE table_name
SET column1 = value1, column2 = value2, ...
WHERE condition;

Parameter Details:

table_name is the name of the table wherein data rests scheduled anticipating functional updates.
column1, column2, … highlight discrete positional columns formally recognized awaiting modifications outright.
value1, value2, … outline structurally novel information intrinsically scheduled effectively overwriting earlier dated legacy materials seamlessly.
WHERE condition represents conditional non-mandatory clauses functionally singling filtering parameters strictly governing explicitly updated rows. Omitting completely identical WHERE clauses subsequently universally upgrades holistically comprehensive tables unconditionally totally.

Complementary Additions:

You remain effectively legally permitted upgrading independently singular and synchronously mixed multifield properties jointly.
You seamlessly retain liberties programming absolutely disparate multifaceted conditions freely enveloped universally underneath WHERE subqueries.
You natively manage comprehensive wide-scale adjustments immediately localized purely isolated targeting standalone tables independently.

Implementing conditional bounds effectively anchored via WHERE structures provides phenomenally irreplaceable instrumental control purposefully identifying discrete unique tabular boundaries specifically designated requiring surgical alterations effectively.

1
2
3
4


-- Update the value of a single column
UPDATE employees
SET salary = 60000
WHERE employee_id = 101;

1
2
3
4


-- Update the values of multiple columns
UPDATE orders
SET status = 'Shipped', ship_date = '2023-03-01'
WHERE order_id = 1001;

1
2
3
4


-- Use an expression to update a value
UPDATE products
SET price = price * 1.1
WHERE category = 'Electronics';

1
2
3
4
5
6
7
8


-- Update using a value from a subquery
UPDATE customers
SET total_purchases = (
 SELECT SUM(amount)
 FROM orders
 WHERE orders.customer_id = customers.customer_id
)
WHERE customer_type = 'Premium';

While authentically negotiating functionally operational structured rational core relational database engines like specifically MySQL inherently, deploying UPDATE statements generally consistently echoes feedback indicating successfully modified elements matching corresponding filtering parameters aligned alongside comprehensive WHERE clauses.

Showcasing structural examples, notably modernizing distinct elements bound identifying primarily tracking id=1 exactly natively:

1
2
3


mysql> UPDATE students SET name='大宝' WHERE id=1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

MySQL distinctly natively signals explicit responsive 1, transparently immediately visible inspecting completely rendered identical results cleanly reading Rows matched: 1 Changed: 1 accurately unequivocally.

Should corresponding updates primarily affect tracking exclusively registering elements possessing intrinsically tracking id=999 similarly:

1
2
3


mysql> UPDATE students SET name='大宝' WHERE id=999;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0

MySQL naturally natively signifies respective conditional returning 0, explicitly transparently visible natively inspecting distinctly echoing outputs explicitly validating Rows matched: 0 Changed: 0 accurately clearly.

DELETE Removal

Basic fundamental grammar orchestrating structured DELETE syntaxes structurally aligns identically resembling:

1

DELETE FROM <Table Name> WHERE ...;

Applying illustrative formatting, attempting functionally eliminating distinct entities harboring unique mapping identifiers securely registered notably id=1 localized strictly natively bounded among active students properties structurally explicitly fundamentally entails encoding practically:

1
2
3
4


-- Delete the record with id=1:
DELETE FROM students WHERE id=1;
-- Query and observe the results:
SELECT * FROM students;

Deliberately paying meticulous attention regarding operational definitions surrounding conditional bounding WHERE elements dynamically filtering out structurally precise parameters mandating prompt executions authentically mirrors exact operations mirroring identical capabilities natively accessible inherently inside structural comparable UPDATE variants. As identically similarly analogous identically dynamically natively, deploying structural functional declarative DELETE operations inherently successfully eliminates identically diverse extensive multiple item datasets independently automatically simultaneously effectively cleanly essentially perfectly similarly entirely efficiently fundamentally:

1
2
3
4


-- Delete records with id=5,6,7:
DELETE FROM students WHERE id>=5 AND id<=7;
-- Query and observe the results:
SELECT * FROM students;

Assuming implicitly applied uniquely distinguishing boundary conditions explicitly failing matching corresponding any active database contents intrinsically perfectly effectively directly completely natively prevents completely DELETE statements automatically triggering system errors, identically mirroring similar outcomes consequently completely yielding total absences involving active eliminations occurring effectively whatsoever natively. Exemplifying analogous parallel dynamically practically:

1
2
3
4


-- Delete the record with id=999:
DELETE FROM students WHERE id=999;
-- Query and observe the results:
SELECT * FROM students;

Finally, it is essential to be extremely careful. Similar to UPDATE, a DELETE statement without a WHERE condition will delete the data for the entire table:

1

DELETE FROM students;

In this case, all records in the entire table will be deleted. Therefore, you must also be very careful when executing a DELETE statement. It is best to first use a SELECT statement to test whether the WHERE condition filters out the expected set of records, and then use DELETE to delete them.

When using a true relational database like MySQL, the DELETE statement will also return the number of deleted rows and the number of rows matching the WHERE condition.

For example, individually executing the deletion of records with id=1 and id=999:

1
2
3
4
5


mysql> DELETE FROM students WHERE id=1;
Query OK, 1 row affected (0.01 sec)

mysql> DELETE FROM students WHERE id=999;
Query OK, 0 rows affected (0.01 sec)

CREATE Creation

MySQL 创建数据表 | 菜鸟教程

1
2
3
4
5
6
7
8


-- User table instance
CREATE TABLE users (
 id INT AUTO_INCREMENT PRIMARY KEY,
 username VARCHAR(50) NOT NULL,
 email VARCHAR(100) NOT NULL,
 birthdate DATE,
 is_active BOOLEAN DEFAULT TRUE
);

Instance Analysis:

id: User id, integer type, auto-incrementing, acting as the primary key.
username: Username, variable-length string, empty values not allowed.
email: User email, variable-length string, empty values not allowed.
birthdate: User’s date of birth, date type.
is_active: Whether the user has been activated, boolean type, default value is true.

The above is just a simple instance utilizing some common data types including INT, VARCHAR, DATE, BOOLEAN. You can choose different data types depending on actual needs.

The AUTO_INCREMENT keyword is deployed for creating an auto-incrementing column, and PRIMARY KEY is used for defining a primary key.

If you desire to assign the data engine, character set, and sorting rules upon creating the table, you may employ CHARACTER SET alongside COLLATE clauses:

1
2
3
4


CREATE TABLE mytable (
 id INT PRIMARY KEY,
 name VARCHAR(50)
) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

Tech on ottercoconut's Blog

Synology NAS Proxy and Emby Configuration

v2rayA

Emby

How to Use Host Proxy on WSL2

.wslconfig

v2rayN

FireWall

wsl2

Cleaning Up Old Settings

~/.bashrc

Solution If It Still Fails

PyTorch Basics

Dataset

Custom Dataset

TensorBoard

SummaryWriter

add_scalar()

add_image()

add_graph()

Common Transforms

ToTensor

Normalize

Resize

Compose

PyTorch Dataset Usage

DataLoader

nn.Module

Convolution Conv

(Max) Pooling MaxPool

Loss Functions and Backpropagation

Optimizer

PyTorch in Practice: CIFAR10

Maven Fundamentals

Reference Websites

Maven Fundamentals

Introduction to Maven

Maven Project Structure

Summary

Dependency Management

Dependency Relationships

Unique ID

Maven Mirrors

Searching for Third-Party Components

Command Line Compilation

Using Maven in an IDE

Summary

Build Process

Lifecycle and Phase

Goal

Summary

Java Multithreading

Reference Websites

Java Multithreading

Process/Thread

Multithreading

Creating Multithreading

Thread Priority

Summary

Thread Blocking

Thread.sleep()

Thread.join()

Summary

Interrupting a Thread

Thread.interrupt

The running Flag

The Purpose of volatile

Summary

Thread State

The System - Five States

Java - Six States

Core Methods in the Thread Class

Daemon Threads

Summary

Thread Synchronization

synchronized Synchronization Lock

Operations That Do Not Require synchronized

Immutable Objects Do Not Require Synchronization

Summary

Thread Synchronization Methods

`Thread.sleep()`

`Thread.join()`

`Thread.interrupt`

The `running` Flag

The Purpose of `volatile`

`synchronized` Synchronization Lock

Operations That Do Not Require `synchronized`

The `synchronized` Modifier

`wait()`

`notify()`