Image-Based Detection

CNNs for Defect Detection

Deep learning, particularly Convolutional Neural Networks (CNNs), has transformed defect detection in semiconductor manufacturing:

• SEM image classification: After inspection tools locate potential defects, a Review SEM captures high-resolution images. CNNs classify these images into defect categories (particle, bridge, scratch, nuisance, etc.) with >95% accuracy, replacing manual human review.
• Wafer map pattern recognition: CNNs classify wafer-level defect patterns (center, edge, ring, scratch, random) to identify root causes. Input: 2D defect density map as an image.
• Object detection: Models like YOLO or Faster R-CNN can locate and classify multiple defects in a single large SEM or optical image.

Common architectures in production:

• ResNet, EfficientNet: Standard backbone networks for classification
• U-Net: For segmentation — pixel-level defect delineation
• Vision Transformers (ViT): Emerging for their ability to capture global context

Pixel-Level Defect Segmentation with U-Net

Classification answers "is there a defect?", but yield engineers also want the exact pixels of the defect — for area measurement, killer-defect screening, and overlap with design layers. U-Net is the standard semantic-segmentation architecture in this space: an encoder–decoder with skip connections that preserves spatial detail.

import torch
import torch.nn as nn
import torch.nn.functional as F

def double_conv(in_ch, out_ch):
    return nn.Sequential(
        nn.Conv2d(in_ch, out_ch, 3, padding=1, bias=False),
        nn.BatchNorm2d(out_ch),
        nn.ReLU(inplace=True),
        nn.Conv2d(out_ch, out_ch, 3, padding=1, bias=False),
        nn.BatchNorm2d(out_ch),
        nn.ReLU(inplace=True),
    )

class UNet(nn.Module):
    """Compact U-Net for binary SEM defect segmentation (defect / background)."""
    def __init__(self, in_channels=1, n_classes=2, base=32):
        super().__init__()
        self.enc1 = double_conv(in_channels, base)
        self.enc2 = double_conv(base, base * 2)
        self.enc3 = double_conv(base * 2, base * 4)
        self.enc4 = double_conv(base * 4, base * 8)
        self.bottleneck = double_conv(base * 8, base * 16)

        self.up4 = nn.ConvTranspose2d(base * 16, base * 8, 2, stride=2)
        self.dec4 = double_conv(base * 16, base * 8)
        self.up3 = nn.ConvTranspose2d(base * 8, base * 4, 2, stride=2)
        self.dec3 = double_conv(base * 8, base * 4)
        self.up2 = nn.ConvTranspose2d(base * 4, base * 2, 2, stride=2)
        self.dec2 = double_conv(base * 4, base * 2)
        self.up1 = nn.ConvTranspose2d(base * 2, base, 2, stride=2)
        self.dec1 = double_conv(base * 2, base)

        self.out = nn.Conv2d(base, n_classes, 1)

    def forward(self, x):
        e1 = self.enc1(x)
        e2 = self.enc2(F.max_pool2d(e1, 2))
        e3 = self.enc3(F.max_pool2d(e2, 2))
        e4 = self.enc4(F.max_pool2d(e3, 2))
        b  = self.bottleneck(F.max_pool2d(e4, 2))

        d4 = self.dec4(torch.cat([self.up4(b),  e4], dim=1))
        d3 = self.dec3(torch.cat([self.up3(d4), e3], dim=1))
        d2 = self.dec2(torch.cat([self.up2(d3), e2], dim=1))
        d1 = self.dec1(torch.cat([self.up1(d2), e1], dim=1))
        return self.out(d1)  # logits, shape: (B, n_classes, H, W)


# Dice loss handles severe class imbalance (defect pixels << background pixels)
def dice_loss(logits, target, eps=1e-6):
    probs = F.softmax(logits, dim=1)[:, 1]            # P(defect)
    target = (target == 1).float()
    intersection = (probs * target).sum(dim=(1, 2))
    union = probs.sum(dim=(1, 2)) + target.sum(dim=(1, 2))
    return 1.0 - ((2 * intersection + eps) / (union + eps)).mean()


def train_step(model, image, mask, optimizer):
    logits = model(image)
    loss = 0.5 * F.cross_entropy(logits, mask) + 0.5 * dice_loss(logits, mask)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    return loss.item()

Why these design choices matter

• Skip connections carry fine-grained pixel locations from encoder to decoder — without them, defect boundaries blur
• Dice loss directly maximises overlap of predicted and true defect pixels; pure cross-entropy collapses to "predict all background"
• Small base channel count (32) keeps the model deployable on inline inspection-tool GPUs (latency budget ~50 ms/image)