Tạo dựng ảnh hiển vi với tốc độ phân giải siêu cao phục vụ cho các ứng dụng khoa học

Tom tat cac ket qua nghien cuu chinh ciia de tai Ket qua ve khoa hgc nhung dong gop cua de tai, cac cong trinh khoa hpc da cong bo De tai da dat duac nhung ket qua chinh ve khoa hoc sau

DAI HQC QUOC GIA HA NOI

Tao dirng anh hien vi vol do phan giai sieu cao phuc vu cho

cac u*ng dung khoa hoc

Ma so: QC.09.07

Chu nhiem de tai: Le Vu Ha

DAI HOr OLIOC C^IA HA NOI

TRUNG 1AM \HON^ ilN IHU VIEN

000600000"^^

HaN6i-2010

MUC LUC

• •

1 Bang giai thich cac chu* viet tat 3

2 Danh sach nhirng ngvoi tham gia thyc hien de tai 3

3 Danh muc cac bang so li^u 3

4 Danh myc cac hinh ve 3

5 Tom tdt cdc ket qua nghien cmi chinh cua de tai 5

6 BAO CAO T 6 N G K £ T 5

6.1 Dat van de 5 6.2 Tong quan cac van de nghien cihi 7

6.3 Muc tieu va Not dung Nghien cihi 8

6.3 L Xay dung thuat toan tao dung anh SRtii cac anh LR 8

6.3.2 Ket qua thuc nghiem va ket luan 15

Ket qua so sanh cac phuong phap n^i suy duo'c the hien trong cac bang sau day, cac

gia tri trong bang la PSNR ciia anh HR duac tai tao lai tu cac anh LR mo phong, so

vol anh HR ban dau 19 Chung ta co the thay rang trong hau het cac tnrong hpp phuong phap KRI deu cho

ket qua tot nhat 20 6.4 Thoi gian, dia diem va phuong phap nghien ciru 20

6.5 Ket qua nghien cuu 20

6.5.1 Cac cong bo lien quan den ket qua cua de tai 20

6.5.2 Ket qua dao tao cua De tai 20

6.5.3 Ket qua ung dung cua De tai 21

6.6 Thao luan 21 6.7 Ket luan va kien nghj 21

6.8 Tai lieu tham khao 21

7 PHU LUC (cac giay t^ gui kem) 23

1 Bang giai thich cac chu viet tat

SR (Super Resolution)

HR (High Resolution)

LR (Low Resolution)

PSF (Point Spread Function)

CCD (Charge Coupled Device)

MAP (Maximum A Posteriori)

MSE (Mean Squared Error)

Do phan giai sieu cao

Do phan giai cao

Do phan giai thap Ham nhoe (ham truyen dat cua thiet bi tao anh) Thi^t bi tich dien kep (kilu cam bien dung trong thiet bi chup anh so)

Hau nghiem cue dai (tieu chuan diing trong tinh toan toi uu/uac luong)

Sai so binh phuomg trung binh

2 Danh sach nhung nguoi tham gia thyc hien de tai

Nguyen Thang Long

Nguyen Hong Thinh

Tran Hoang Tung

Nguyen Viet DQng

Nguyen Thuy Hoang Phong

Hoc vj

Tien sT Tien sT Thac sT Thac sT Cunhan Cunhan

Caquan cong tac

Bo mon Xu ly thong tin

Bp mon Vi co dien tu

va Vi he thong

Bp mon Xu ly thong tin

Bp mon Xu ly thong tin

Bp mon Xu ly thong tin

Bp mon Xu ly thong tin

Nhiem vu

Chu nhiem de tai Thanh vien Thanh vien Thanh vien Thanh vien Thanh vien

3 Danh muc cac bang so lieu

Bang 1: PSNR ciia cac anh HR tai tao tu chuoi anh Lena 19

Bang 2: PSNR cua cac anh HR tai tao tir chuoi anh Vase 19

Bang 3: PSNR cua cac anh HR tai tao tir chuoi anh Apaca 20

4 Danh muc cac hinh ve

Hinh 1: Mo hinh qua trinh tao anh 8 Hinh 2: Lien he giua anh dp phan giai cao va cac anh dp phan giai thap 9

Hinh 3: Vi du ve lay mau tren tin hieu 9 Hinh 4: Qua trinh tao dung anh SR tir tap anh LR 10

Hinh 5: Trai: mpt anh LR trong chuoi anh Apaca, Phai: anh SR (dp phan giai tang 5x5 l^in)

15 Hinh 6: Trai: mpt anh LR trong chuoi anh Emily Phai: anh SR (dp phan giai tang 5x5 1^)

15

Hinh 7: Trai: mpt anh LR trong chuoi anh Text Phai: anh SR (dp phan giai tang 5x5 ikn)

Hmh 8: Trai: mpt anh LR trong chuoi anh Vase Phai: anh SR (dp phan giai tang 5x5 \kn)

Hinh 10: Hai anh trong mpt chu6i anh chup bang thiet bi hien vi 17 Hinh 11: Cac anh SR hien vi duac tao dung tir chudi anh LR 18

5 Tom tat cac ket qua nghien cuu chinh ciia de tai

Ket qua ve khoa hgc (nhung dong gop cua de tai, cac cong trinh khoa hpc da cong bo)

De tai da dat duac nhung ket qua chinh ve khoa hoc sau day:

• Nghien cuu cac phuang phap tao anh voi dp phan giai sieu cao tir nhieu anh a dp phan giai binh thuong, ap dung cho anh hien vi quang hpc

• Nghien cuu so sanh cac phuang phap noi suy khac nhau su dung trong qua trinh tao dung anh sieu phan giai

• Danh gia dp chinh xac ciia cac phucmg phap tao dung anh dugc ap dung

Mot phan trong cac ket qua tren da duac cong bo trong bao cao khoa hoc sau day:

Hong-Thinh Nguyen, Viet-Dung Nguyen, Ha Vu Le, "Super-Resolution Image

Construction from High-Speed Camera Sequences", Proceedings of the

International Conference on Advanced Technologies for Communications., Ho Chi

Mmh City, October 2010

Ket qua phuc vu thuc te (cac san pham cong nghe, kha nang ap dung thuc te)

• Chuang trinh cai dat tren Matlab ap dung dk tao anh dp phan giai cao tir chudi cac

anh dp phan giai thap hoac video dp phan giai thap

Ket qud dao tao (so lupmg sinh vien, so lugng hpc vien cao hpc, nghien cuu sinh tham gia

thuc hien lam viec trong de tai, so khoa luan, luan van da hoan thanh va bao ve)

• 02 sinh vien dai hpc va 02 hpc vien cao hpc da tham gia lam viec va nghien cuu

trong khuon kh6 dk tai

• 01 khoa luan tot nghiep dai hpc va 01 luan van cao hpc theo huong cua de tai da bao

ve thanh cong

• 01 cong trinh NCKH cua sinh vien tham gia de tai dat giai nhat c4p truang va giai khuyen khich cap Bp GD-DT

Ket qud nang cao tiem lire khoa hpc (nang cao trinh dp can bp va trang thiet bi hgc phhn

mem da xay dung dugc giao nop dua vao su dung tai don vi)

• De tai da gop phan phat trien mot bp thu vien phan mem xir ly anh hieu qua cho phep tiep tuc dugc khai thac tiep cho phuc vu nghien cuu va dao tao

• De tai da tang cuang kien thuc cho cac can bp cua bp mon ve xu ly tin hieu, xu ly anh va video

6 BAO CAO TONG KET

6.1 Dat van de

Dp phan giai ciia hinh anh la mpt chi tieu chat lugng quan trpng ciia cac thilt bi chup anh

so, dugc do bang mot dai lugng ty le nghich voi kich thuac moi diem anh tren be mat cua cam bien hinh anh dimg trong thiet bi chup anh so Voi viec cac thiet bi hiln thi hinh anh

thilt bi chup anh ciing c ^ phai dugc nang cao mpt each tuong irng de tan dung dugc nang luc ma cac cong nghe hiln thi hinh anh mang lai Tuy nhien, neu luon thay the cac thiet bi

chup anh dang dugc su dung rit rpng rai trong rat nhieu Imh vuc, tir cac lihh vuc khoa hpc

— cong nghe cho toi cac Imh vuc giai tri, bang cac thiet bi co dp phan giai cao han thi su lang phi se rat Ion vi loai thiet bi nay co gia thanh rat cao, nhat la vai cac thiet bi chup anh chuyen dung su dung trong cong nghiep ciing nhu trong nghien cuu khoa hpc Do do, ton tai mpt nhu cau thuc te cho cac cong cu xu ly anh cho phep tao dung nhung buc anh co chat lugng tuang ducmg san pham cua cac thiet bi chup anh co dp phan giai cao tir anh dugc chup bdi cac thiet bi co dp phan giai thap han

Tao anh sieu phan giai tir cac anh co dp phan giai thap la van de da dugc nghien cuu tir nhieu nam va van tiep tuc la chu de co tinh thai sir trong cong dong xir ly anh Thuat ngu

"sieu phan giai" dugc hieu vai nghia dp phan giai cua anh dugc tao ra cao han dp phan giai thuc cua thiet bi chup anh Hang tram phuang phap tao anh sieu phan giai da dugc de xuat, voi hang chuc phuang phap da tra thanh kinh dien (Ur&Gross, 1992; Tekalp et al, 1992; Komatsu et al, 1993; Hardie et al, 1998; Baker&Kanade, 1999; Elad&Feuer , 1999; Nguyen&Milanfar, 2000; Altunbasak&Patti , 2000; Joshi&Chaudhuri, 2002; Protter et al, 2009, ) Rieng trong nam 2010 so lugng bai bao lien quan tai van de nay cong bo tren hai tap chi CO uy tin nhat trong Imh vuc xur ly anh la IEEE TPAMI va IEEE TIP cung rat nhieu (Yang et al, 2010; Robinson et al, 2010; Gajjar&Joshi, 2010; Kim&Kwon, 2010; Xiong et

al, 2010; Belekos et al, 2010; Yuan et al, 2010; van Eekeren et al, 2010; Mallat&Yu, 2010), The nhung trong thuc te cac ky thuat tao anh sieu phan giai, nhat la cac phuang phap tir nhieu anh, van rat it dugc ung dung Ly do chinh la boi vi cac phuang phap da dugc de xuat mac du co ca sa ly thuyet virng chac va cac ket qua thuc nghiem tot van that bai khi ung dung trong dieu kien thuc, Mpt so nguyen nhan sau day co thi ly giai dieu do:

• Phan Idn cac phuang phap tao anh sieu phan giai tir nhilu anh doi hoi phai xac dinh dugc cac dich chuyen xay ra giua cac anh vdi dp chinh xac cao (muc dudi diem — subpixel), dd la van de rat kho khi cac chuyin dpng xudt hien trong chu§i anh rdt phuc tap hoac khi anh bi nhieu

• Trong dieu kien thirc cac anh thu dugc tir video camera co dp phan giai thdp thudng

bi nhoe do nhieu ly do ket hgp nhu do camera co tieu cu co dinh, do anh hudng cua rung dpng trong moi trudng, va do chuyen dpng cua cac doi tugng nen g ^ nhu khong thi xac dinh chinh xac ham nhoe cua qua trinh chup anh, trong khi hiu hit cac phuang phap tao anh da phan giai bo qua anh hudng cua anh nhoe hoac gia thilt ham nhoe ciia he thong cd the xac dinh duac

Muc tieu cua chiing toi trong de tai nay khong phai la de xuat mpt phuang phap tao anh da phan giai mdi ma la tim mpt linh vuc ung dung trong thuc tl d do cac phuang phap da cd

cd thi thuc su phat huy tac dung Vdi muc tieu nhu vay, chiing toi da chpn nghien cuu kha nang ung dung ky thuat tao anh sieu phan giai tir nhieu anh cho anh hiln vi quang hpc (optical microscopic images) vi nhirng ly do sau day:

• Mac du cac thiet bi hien vi quang hpc thudng co dp phong dai rat Idn va six dung cac thilt bi chup anh cd dp phan giai cao, nhu cau co dugc hinh anh vdi dp phan giai cao

hon niia luon luon dugc mong muon, nhat la trong cac Itnh vuc nghien cuu ve cac cau tnic cd kich co d muc nano

• Hinh anh chup bang thiet bi hien vi quang hpc thudng cd chat lugng rat cao do dieu kien chup anh thudng dugc kiem scat tot: dilu kien chieu sang tot, it nhieu, it nhoe Trong trudng hgp can thiet viec do dac ham nhoe ciia qua trinh chup anh ciing cd the thuc hien dugc kha dl dang

• Viec tao ra chuoi anh vdi cac dich chuyen nho giiia cac anh co the dugc kiem soat de dich chuyen giira cac anh nay kha don gian va de dang tinh toan/do dac dugc vdi dp chinh xac cao

Ky thuat tao anh sieu phan giai neu ap dung thanh cong cho cac anh hien vi se la mpt cong

cu hiru dung phuc vu cho cac phong ihi nghiem nghien cuu va phan tich trong cac lihh vuc nhu vat lieu, y-sinh hpc va y te

6.2 Tong quan cac van de nghien cuu

Cd the chia cac phucmg phap tao anh sieu phan giai thanh hai nhdm: cac phuang phap thuc hien tir duy nhat mot anh, thudng gpi la noi suy anh, va cac phuang phap thuc hien tir nhieu anh U'u diem ciia cac phuang phap noi suy anh la chiing cd dp phuc tap tinh toan thap hon

so vdi cac phuong phap thuc hien tir nhieu anh Tuy nhien, nhieu chi tiet bi mat tren anh cd

dp phan giai thap do hien tugng aliasmg se khdng the khdi phuc lai dugc neu chi sir dung mpt anh, hoac ngugc lai, cac phuang phap npi suy hay tao ra cac chi tiet khong cd thuc (artifacts), vi vay kho cd thi tao anh sieu phan giai vdi dp chinh xac cao bang cac phucmg phap npi suy don thuin Trong khi do, anh hiln vi sir dung trong nghien cuu hay phan tich can phai cd dp chinh xac cao vi chiing dugc diing khong chi de quan sat ma con cho cac phep do dac kich thudc cac cau tnic Do dd, cac phuang phap tao anh sieu phan giai tir nhieu anh se phii hgp hon khi ap dung cho anh hiln vi vi cac phuang phap nay su dung dugc nhieu thong tin chi tiet hon tap hgp tu nhieu anh de tong hgp thanh buc anh cd dp phan giai cao

Qua trinh tao anh sieu phan giai tir nhieu anh thudng bao gom cac budc nhu sau:

1 Uac lugng sir dich chuyen giua cac anh nham muc dich bii cac dich chuyin dd dl

dang ky cac anh dau vao (gpi la cac anh co dp phan giai thap - LR) len mpt ludi dilm cd dp phan giai cao hon Vi vay cac dich chuyen ciia moi dilm giiia hai anh lien tiep can dugc xac dinh vdi dp chinh xac tuong duong vdi ludi dilm cd dp phan giai cao (HR), nghia la vdi dp chinh xac d muc dudi kich thudc dilm cua cac anh ban dau, Vdi cac chudi anh co chua nhieu chuyin dpng phuc tap, dd la mpt nhiem

vu rat khd khan Tuy nhien, vdi cac he thdng chup anh tinh vi nhu cac thilt bi hiln vi quang hpc, viec chii dpng dieu khien de cac chuyen dich giiia cac anh rat nho va don gian la kha de dang, nen viec xac dinh chinh xac cac chuyin dich do co thi thuc hien vdi nhiing phuong phap dcm gian

2 Dien dir lieu tir cac anh LR (sau khi da dugc bii dich chuyen) vao ludi HR

3 Ndi suy cac dilm cdn thieu trong ludi HK Viec phai dimg den ndi suy la khdng thi

tranh khdi khi chudi anh LR chiing ta cd khdng dii de dien vao th ca cac dilm trong

ludi HR Cd rat nhilu phuang phap ndi suy anh cd thi ap dung dugc, tir cac phuong phap ndi suy toan hpc thudng dimg, ndi suy tin hieu bang ham sine, cac phuang phap thdng ke, den cac phuong phap ndi suy su dung cac dac tnmg hinh hpc ciia anh Dilu quan trpng d day la lira chpn dugc phuang phap ndi suy thich hgp, dam bao tinh chinh xac cao ciia cac chi tiet dugc ndi suy

4 Khu nhoe (deblurring): mpt sd v4n de xay ra trong qua trinh chup anh cd the gay ra hien tugng nhoe lam md di cac chi tiet tren anh, nhu nhoe do dat sai tieu cu hay nhoe do rung dpng cua mdi trudng anh hudng tdi thilt bi Vdi cac he thdng chup anh hien vi day thuc ra khdng phai la van de Idn vi: a) khoang each chup thudng la cd dinh nen cd the chi can hieu chinh he thdng mpt lan de cd dp net cao nhat cho ca chudi anh, va b) cac he thdng chup anh hien vi thudng dugc thiet ke cd kem theo he chdng rung de han che anh hudng tir rung dpng trong mdi trudng Ngoai ra, neu can xac dinh chinh xac ham nhoe cua qua trinh chup anh de thuc hien viec khu nhoe, vdi cac he thdng hien vi ham nhoe thudng do dugc mpt each kha chinh xac (do ham nhoe ciia he thdng chup anh chinh la dap ung xung cua he thdng, nen vdi he thdng chup anh hien vi ngudi ta thudng do ham nhoe bang each chup mpt diem sang rat nho tren mat phang dat vat mau ciia kinh hien vi, tao ra bdi lazer, trong dieu kien hoan toan khdng cd anh sang) Khi do cd the ap dung nhieu phucmg phap khir nhoe vdi ham nhoe da biet

Cac phuong phap hien co quan tam chu yeu tdi viec cai thien chat lugng hinh anh theo cam quan ciia con ngudi hon la tdi dp chinh xac ciia hinh anh, vi vay khdng dam bao viec nang cao chat lugng hinh anh cung se giiip nang cao dp chinh xac cua cac phep phan tich va do dac tren anh Cac de tai nghien cuu ve tao anh sieu phan giai vdi ddi tupmg chinh la anh hien vi quang hpc ciing rat it

6.3 Muc tieu va Noi dung Nghien cuu

Muc tieu ciia de tai la nghien cuu cac giai thuat tao dung anh sieu phan giai tir nhilu anh nhim dua ra mpt giai phap ung dung cho anh hien vi quang hpc Chiing tdi chi tap tnmg vao cac giai thuat udc lugng dich chuyen giiia cac anh va cac giai thuat ndi suy anh do nhiing ly do da dugc trinh bay d phan tren Cac ndi dung nghien ciru dugc trinh bay cu thi trong cac muc dudi day

6.3,1, Xay dimg thuat todn tao dung dnh SR tir cdc dnh LR

Mo hinh bdi todn

Ban chit ciia qua trinh chup anh la qua trmh lay mau khdng gian lien tuc trong do cam biin

ciia thiet bi anh ddng vai trd nhu mpt bp lay mau tin hieu hai chieu trong mat p h ^ g anh Anh thu dugc se la ma tran cac gia tri thi hien muc sang toi thu dugc tir cam biin anh tiiy vao dp xa gan, mau sac ciia vat the Ve nguyen tac, khi tan sd lay mau cang Idn (nghia la kich thudc diem anh tren cam bien hinh anh cang nhd), anh thu dugc se cang ro net; va khi tan sd lay mau dii Idn (theo tieu chuan Nyquist), anh thu dugc se the hien day du thdng tin ciia khdng gian lien tuc va mat thudng se khdng phan biet dugc su rdi rac cua thdng tm Ta

dinh nghia anh nay la anh cd do phan gidi cao ly tuang Tuy nhien, anh thuc ma chiing ta

thu dugc qua qua trmh chup anh thudng cd dp phan giai thap hon dp phan giai ly tudng dd

Ngudi ta thi hien dilu do bSng mot su gidm toe do lay mau (dovm-sampling), thudng gpi

tat la giam tdc, nghia la giam dp phan giai cua anh Ngoai ra, anh chup dugc cdn la ket qua cua viec biin doi hinh anh ly tudng cua khung canh thuc bdi qua trinh chup anh, dai dien

bdi dap ung xung ciia qua trinh nay, thudng dugc gpi la ham nhoe (point spread function)

Md hinh qua trinh tao anh dugc minh hpa trong hinh dudi day:

ANH O O PHAN CIAI THAP

Hinh 1: Mo hinh qud trinh tao dnh

De cd the xay dung thuat toan tao dung anh sieu phan giai (SR), ngudi ta cd gang md hinh

hoa mdi lien he giiia anh cd dp phan giai cao (HR) ly tudng va anh cd dp phan giai thip

(LR), va mdi lien he giua cac anh LR vdi nhau Neu md hinh dugc xay dung la chinh xac,

b ^ g each dao ngugc lai qua trinh tir anh HR->LR, chiing ta hoan toan cd thi tao dung lai anh HR, tuc la anh SR, tir cac anh LR

Gpi {¥*} la tap cac anh LR, chup bdi cung mpt may anh, trong cimg dieu kien, nhung cd

su khac nhau do sir dich chuyen ciia camera hay ddi tugng dugc chup trong qua trinh chup anh Gpi X la anh dp phan giai cao ly tudng ma chiing ta mong mudn co dugc Anh LR thu dugc cd thi coi nhu ket qua ciia qua trinh chup anh bdi he thdng cd ham nhoe h va sau do giam tdc tir anh dp phan giai cao ly tudng Qua trinh chup anh cdn cd thi bi anh hudng ciia nhieu tir mdi trudng, Neu ky hieu D va H lan lugt la toan tir giam tdc va nhan chap vdi h,

va n dai dien cho thanh phan nhieu, chiing ta cd: Y^ = DHXt+n, d do X* la mpt phien ban bi

dich chuyen (tinh tien va xoay) ciia X Gpi F^ la toan tii dich chuyen tuong ung, chiing ta cd dugc md hinh toan hpc ciia qua trinh tao anh dugc bieu dien bang phuong trinh dudi day:

Yk = DHXi+n = DHF;tX+n

ANH D O PWAN CIAI THAP YK

GIAMTOC

^

^^it^S/-^*• W A N ClAl CAO NHUNC 81 N H 6 E

i i - i M ^ _ „ *&

Hinh 2: Lien he giira dnh do phan gidi cao vd cdc dnh do phan gidi thap

Ca sa cua thuat todn

Gia sir rang sir dich chuyen giua cac hinh anh la hoan toan ngau nhien Cau hoi dat ra la trong dieu kien nao chiing ta cd the ket hgrp cac anh LR de thu dugc anh cd dp phan giai cao hon

De don gian chiing ta cd the xem xet viec lay mau tin hieu mot chieu

1 M+*iM+*i

/ ( / ) : Tin liicu lien luc

1/0 C*^) : La\ m5u tai (-0

y i (•"•) : L5y mau tai t-tl

J V - l i V - 1

Hinh 3: Vi du ve lay mau tren tin hieu

Tin hieu rdi rac dugc lay mau (sampled) tir tin hieu lien tuc Ta cd the dimg khai niem dp phan giai cua tin hieu dl chi mat dp mau tin hieu hay tan sd lay mau, dai lugng nay ty le nghich vdi chu ky lay mau (khoang each giira hai mau) Ddi vdi anh sd, mpt chu ky liy mlu (hai chieu), tuong duong vdi kich thudc mpt diem anh

Gia sir chiing ta liy miu mpt tin hieu lien tuc, sau do dich tin hieu dd di mpt khoang nhd va lay mau lai vdi cimg t4n sd lay mau trudc dd Dieu dd ciing tuong duong vdi viec lay mau mpt tin hieu lien tuc hai lan vdi cimg tin sd lay mau nhung lech pha (khac nhau dilm gdc)

nhu dugc minh hpa trong Hinh 3 De thay rang hai tap diem cua hai lan lay mau nay chi

khong triing lap khi khoang dich khong phai Id mot so nguyen lan cua chu kx lay mau Neu

ket hgp hai tap mau vdi nhau, chiing ta se cd dugc mpt tin hieu lay mau tir tin hieu lien tuc ban dau vdi mat dp mau cao hon tan sd lay mau thuc, tuy nhien chua the coi dd la tin hieu

cd dp phan giai cao hon vi khoang each giiia cac mau cd thi khdng deu Budc can thiet tiep theo la ndi suy tir tap mau phan bd theo mpt ludi khdng deu dl smh ra mpt tin hieu SR theo mpt ludi deu Dieu ch4c chdn la tin hieu SR nay se cd dp chinh xac cao hon tin hieu dugc ndi suy chi tir mot tap mau ban dau vi sd diem mau dugc dimg de ndi suy Idn hon

Ap dung cho anh hien vi quang hpc, chiing ta cd the su dung phuong thuc nhu tren de tao anh sieu phan giai vdi cac dieu kien sau day;

• Cd dugc nhieu anh chup cimg mpt vat mau bdi cimg mpt he thdng hien vi vdi dieu kien chup anh khdng thay ddi

• Cd sir dich chuyen ciia vat mau giira cac anh vdi khoang dich la ngau nhien hoac dugc dieu chinh de khdng bang mpt sd nguyen lan kich thudc diem anh

Gidi quyet bai todn

Cac budc cua qua trinh tao anh SR dugc minh hpa bang Hinh 4

L/6C LL'OKC DICH CHliVfN

i l N H 6 E , NHlEU MAP, MSE

ANH £>0 PHAN ciAt CAO

Hinh 4: Qud trinh tao dung dnh SR tic tap dnh LR

Nhu da xac dinh trong phan muc tieu, d de tai nay chiing tdi chi tap trung vao hai budc: udc lugng dich chuyen va ndi suy anh

Bir&c 1: Lf&c lirpmg sir dich chimin giira cdc dnh

Cac thuat toan udc lugng chuyen ddng (motion estimation) von da dugc nghien cuu tir rit liu trong nhieu ung dung, nhit la trong cac ky thuat nen video Trong bai toan tao dung anh

SR tir nhilu anh, viec udc lugng dich chuyin phai dat dp chinh xac tucrng duong kich thudc dilm trong ludi HR, tuc la muc dudi dilm (subpixel) tren cac anh LR Mpt sd phuong phap udc lugng dich chuyin vdi dp chinh xac d muc dudi dilm anh da dugc xem xet, bao gdm:

• Phuang phap udc lugng theo khdi anh

• Phuang phap su dung mien tan sd

• Phuang phap sir dung optical flow

Cd thi tdm tit mpt vai net chinh ve cac timg loai phuong phap nay nhu sau:

/, Phuang phdp uac luang theo khoi dnh

Vdi loai phuang phap nay, anh dugc chia thanh cac khdi (block) cd kich thudc cd dinh hay thay doi Sir dich chuyen giiia hai anh dugc tinh toan cho mdi khdi bang each tim kiem vi tri mdi trong anh thu hai cua cac khdi trong anh thir nhit So sanh mpt khdi trong anh thu nhat vdi cac khdi lan can nd trong anh thu hai se cho chiing ta biet khdi gidng nd nhit, tu

do xac dinh dugc dp dich ciia mpt khdi giiia hai anh, Mpt sd dp do cd the dirng dh do sir gidng nhau giiia hai khdi, nhu tdng cdc sai khac tuyet doi (sum of absolute differences — SAD), sai so binh phuang trung binh (mean squared error - MSE), hay tuang quan cheo

chuan hoa (normalized cross correlation - NCC):

chuyen d sao cho SAD(d) nhd nhit, hoac MSE(d) nhd nhit, hoac NCC(d) Idn nhit Cd thi thay d d day tuong ung vdi mpt su dich chuyen d muc diem anh De dat dugc muc chinh xac dudi diem, chiing ta phii ndi suy gia tri cua cac ham dp do ndi tren cho cac gia tri ciia d

d muc dudi diem

Cac khdi trong mpt anh cd the lap lai chinh chiing khdng phai do dich chuyin, vi vay kit qua ciia viec xac dinh dich chuyen ciia cac khdi do vi vay cd the hoan toan sai Ndi chung khi kich thudc cac khdi cang Idn thi kha nang lap lai cang nhd nen chiing ta cd thi lam giam kha nang xay ra cac sai sdt loai dd bang each tang kich thudc khdi Tuy nhien, khi sir dich chuyin khdng hoan toan ddng nhit cho tat ca cac diem trong anh thi su dung khdi cd kich thudc cang Idn se cang lam tang sai sd tin toan dich chuyen cho mdi diem

\ \ r

ii Phuang phdp sic dung mien tan so

Phuong phap su dung miln tin sd dua vao tinh chat vl biin doi Fourier ciia mot tin hieu bi

dich Gia su/i(x, y) la ham bilu dien mpt tin hieu trong khdng gian hai chieu (anh) va^(A:,

y) la/i(x, y) dich di mpt khoang xac dinh bdi vector (xo, >'o):

Gpi Fi{u, v) and F2(u, v) lin lugt la pho Fourier c\mf\(x, y) v^fiix, y) Quan he giira hai pho

nay dugc biet nhu sau:

F^{u,v)=F,{u,v)e'^"'^^'^^^

Sir dich pha giira 2 anh cd the tinh dugc nhu sau:

.^F,(U,V)F;{U^V) ^^.J^^^^^y^)

' \F,(KV)F;{U,V)\

Bien ddi Fourier ngugc cua D(u, v), ky hieu la d{x, y), chinh la ham xung delta tai diem (xo,

yo), tir dd chiing ta thu dugc vector dich chuyen

Phuong phap nay cho phep dat dp chinh xac tdi muc dudi diem ma khdng cin ndi suy, tuy rihien se khdng sir dung dugc trong trudng hgp su dich chuyen khdng hoan toan ddng nhat cho tit ca cac diem trong anh

/// Phuang phdp sir dung optical flow

Optical flow ciia mot chudi inh la mpt trudng vector (vector field) the hien su chuyen dich (hudng va van tdc) tai timg dilm trong moi anh vdi dp chinh xac tdi muc dudi diem Vi vay, su dung optical flow cho phep bieu dien mpt each chinh xac cac chuyen dich trong anh, ke ca trong trudng hgp su dich chuyen khdng hoan toan ddng nhat cho tit ca cac diem

Cd nhieu phuong phap da dugc dua ra de tinh optical flow cua chudi anh Mpt trong nhirng phuang phap don gian va hieu qua nhat la phuang phap cua Lucas va Kanade (Lucas&Kanade, 1981), theo dd vector chuyen ddng tai moi diem dugc xac dinh dua tren dao ham theo thai gian ciia gia tri dilm anh Phuong phap do cd the dugc trinh bay nhu sau:

gia thiet/(x, y) =f(x, y, t) \afi{x, y) =f{x, y, /+5) lin lugt la hai ham bieu dien hai anh lien tiep chup tai hai thai diem t va /+5, ddng thai gia thiet diem cd vi tri tai (x, y) trong anh thu nhit bi dich di mpt khoang xac dinh bdi vector (&, 5^) khi tdi anh thir hai, nghia la:

Theo cdng thuc ciia khai trien Taylor, chiing ta cd:

dx 6i dy 6f dt

Vi Sx/St = Vx va 5y/Si = Vy lin lugt la hai thanh phin cua vector chuyen ddng tai diem (x, y),

chiing ta cd dugc phuang trinh cua optical flow giiia hai anh:

df_ df_ ^ _ 9 /

dx^' dy""' dt

Cac phuong phap xac dinh dich chuyen dua tren dao ham theo thdi gian ciia gia tri diern anh nhu phuong phap Lucas-Kanade tren day chi cd thi ap dung khi dp dich chuyen la rit nhd (d muc dudi diem), nhung ciing do vay cac phuang phap nay de bi anh hudng bdi nhieu Ddi vdi anh hien vi, dieu khien sir dich chuyen de dp dich giira cac anh lien tiep rit nhd d muc dudi diem la kha de dang, va nhieu khdng phii vin de Idn, vi vay phuong phap

su dung optical flow la mpt giai phap thich hgp

Bir&c 2: Ghep dnh vd Npi si^

Budc tiep theo, sau khi da cd dugc nhiing thdng tin ve sir dich chuyen giira cac anh, la dua tit ca cac anh ve cimg mpt ludi HR bang each bii lai dich chuyen cua chiing so vdi cimg mpt gdc, Tir tit ca cac anh LR sau khi da dugc bii dich chuyen, chiing ta se thu dugc mpt tap diem cd vi tri hoan toan ngau nhien trong ludi HR (do sir dich chuyen giiia cac anh la ngau nhien) vdi gia tri mdi dilm la mot gia tri diem anh trong anh LR Tai nhimg vi tri trong ludi HR cd nhieu hon mpt diem tir cac anh LR cimg rod vao, gia tri dugc dien vao se

la gia tri trung binh cua cac diem anh do Ngugc lai, cd nhiing vi tri ciia ludi HR khdng cd diem anh nao roi vao, khi do ndi suy cin dugc ap dung de dien vao cac vi tri chua cd gia tri Chiing ta cd hai each dl npi suy gia tri cac dilm anh trong ludi HR:

• Cach thu nhit: ndi suy tir mdi anh LR tat ca cac gia tri cac diem anh cua ludi HR, sau dd liy trung binh cua cac gia tri dugc ndi suy de cd mpt anh SR duy nhit Vdi each nay, phuong phap ndi suy dugc su dung se la phuang phap ndi suy tir ludi dilm deu (uniform interpolation), tuy nhien dp phuc tap tinh toan se Idn vi mdi diem trong ludi HR deu dugc ndi suy nhieu lin

• Cach thijf hai: chi ndi suy cac vi tri chua cd gia tri trong ludi HR mpt lin tir cac gia tri da dugc dien vao ludi lay d cac anh LR Trong trudng hgp nay chiing ta thudng phai su dung phuang phap ndi suy tir ludi dilm khdng deu (non-uniform interpolation) do vi tri cac diem can ndi suy trong ludi HR la ngiu nhien, tuy nhien khdi lugng tin toan cd the giam so vdi each thu nhit do viec ndi suy chi can thuc hien mpt lin cho mdi diem

Cd rat nhilu phuong phap ndi suy anh, tuy nhien chiing tdi chi quan tam tdi mpt sd phuang phap phu hgp vdi hai tieu chi sau day: cd the thuc hien ndi suy tir ludi dilm khdng deu va kha don gian ve cai dat Hai tieu chi nay dugc dat ra vi muc tieu xa hon ciia chiing tdi la phat triln cac ung dung thdi gian thirc tren cac phin cung xu ly chuyen dung, nen cac giai thuat phai dcm gian va cd dp phuc tap tinh toan thip Ngoai ra, chiing tdi ciing chpn cac phuang phap ndi suy thudc ve cac Idp phuang phap khac nhau nham so sanh hieu qua cua cac Idp phucmg phap khi ap dung cho van de cua de tai

Cac phuong phap ndi suy dugc chiing tdi thii nghiem bao gdm:

• Ndi suy tuyIn tinh (linear interpolation): diy la phuong phap don gian nhit, tuy nhien cd hieu qua kha tot khi su dung dl ndi suy cac tin hieu cd dang tron tru, vi vay phuong phap ndi suy tuyen tinh vin la mpt trong nhiing phuang phap phd bien nhat trong ndi suy tin hieu va anh Theo phuang phap nay, gia tri cho mdi diem anh cin ndi suy trong ludi HR la gia tri trung bmh cd trpng sd cua cac diem trong ludi HR

dugc dien vao tir cac anh LR, d dd mdi trpng sd tuang (mg vdi mdi diem cua anh LR

ty le vdi khoang each tir dilm dd tdi dilm can ndi suy trong ludi HR Phuong phap ndi suy tuyIn tinh su dung dugc khi ndi suy tir ludi dilm khdng deu

• Ndi suy cac diem rai rac su dung ludi tam giac (triangulation-based scattered data

interpolation) (Amidror, 2002): cac phucmg phap ndi suy dii lieu rai rac (scattered data interpolation) la cac phuong phap ndi suy tir ludi diem khdng deu va ludi diem sau khi dugc ndi suy cung khdng nhit thiet phai deu Nguyen tac chung ciia cac phuong phap nay la tao ra mpt ludi diem xip xi mpt be mat (ham) tron tru tir cac diem da cd cua cac anh LR Ludi xip xi nay thudng la mpt ludi tao tir cac mat tam giac (triangle mesh), phuang phap tao ludi tam giac xap xi be mat tir mpt tip diem thdng dung nhit la phuong phap cua Delaunay (Delaunay triangulation) Sau dd, gia tri cho cac diem can dugc ndi suy se dugc tinh tir gia tri tai vi tri cac dinh ciia mat tam giac bao quanh nd, thudng la bang cdng thuc ndi suy tuyen tinh

• Ndi suy bing kernel regression (kernel regression interpolation - KRI) (Takeda et al, 2007): KRI la phuong phap ndi suy dua tren viec xip xi ham tir tap diem da cd tir

cac anh LR bing each xac dinh mpt ham kernel tai mdi dilm da cd va ap dung ham

kernel do cho cac diem cin npi suy lin can nd Phuang phap nay dua tren gia thiet rang mpt sd dac tnmg cua ham cin xap xi ludn bit bien trong mpt pham vi cue bd nhd Vdi anh, do la su lap lai eiia cac cau tnic hinh hpc (geometric regularities) Ndi suy anh dua tren gia thiet ve sir lap lai ciia cac ciu tnic hinh hpc la mpt hudng thu hiit dugc nhieu su quan tam trong vai nam gin diy va cac phuang phap theo hudng nay da chung to dugc sir uu viet so vdi cac phuong phap ndi suy anh co diln KRI la mpt phuong phap trong lap nay, uu diem cua KRI so vdi cac phuang phap khac cimg Idp la KRI cd the dp dung cho ludi khdng deu

6.3,2, Ket qud thirc nghiem vd kit luan

Thuc nghiem dugc thuc hien thanh hai phin rieng biet:

Phan 1: Thuc hien tren tip anh LR md phdng (tir mpt anh cd dp phin giai cao, thuc hien

cac budc giam tdc, dich, cdng nhieu de thu dugc mot tip anh cd dp phin giai thip hon) dugc nhieu tac gia su dung de danh gia cac giai thuat SR Muc dich cua phan nay la so sanh hieu qua cac phuong phap ndi suy khac nhau khi ap dung vao viec tao anh SR Cac tip anh

LR dimg trong phan nay liy tai dia chi: datasets.html

http://users.soe.ucsc.edu/-milanfar/software/sr-Phan 2: Thuc hien tren anh chup true tiep tir kinh hien vi quang hpc, nham minh hpa cho

hieu qua ung dung ma de tai nham tdi

Dudi day la hinh anh cua dir lieu va kit qua thuc nghiem:

Iffiiilii

j?3^'# «;^

Hinh 7: Trdi: mot dnh LR trong chuoi dnh Text Phai: dnh SR (do phan gidi tang 5x5 Idn)

Hinh 8: Trdi: mot anh LR trong chuoi anh Vase Phai: dnh SR (do phan gidi tdng 5x5 Idn)

DAI HQC QUOC GIA HA NQl TRUNG lAiVl [MOi\G IIN T H U V I E N

Hinh 9: Trdi: mot dnh LR trong chuoi dnh Lena Phai: dnh SR (do phan gidi tdng 5x5 lan)

//?«/; 70.* Hai dnh trong mot chuoi dnh chup bdng thiet bi hien vi

-4

W\

Hinh U: Cdc dnh SR hien vi duac tao dung tic chuoi dnh LR

Kit qua so sanh cac phuong phap ndi suy dugc the hien trong cac bang sau diy, cac gia tri

trong bing la PSNR cua anh HR dugc tai tao lai tir cac anh LR md phdng, so vdi anh HR

Scattered 28,30 28,98 30,32 30,58 35,68 31,11

KRI 30,24 30,44 33,24 31,50 36,56 31,58 ]

Bdng 7; PSNR cua cdc dnh HR tdi tao tic chuoi dnh Lena

1 24,78

! 31,43

27,01 35,23 29,79

KRI 29,08 26,28

1 35,62

• 27,03

38,01 31,23

Scattered 27,87 29,31 29,81 28,84 32,91 29,31

KRI 1

29,20 29,92 31,02 28,63 34,78

29,92 Jj

Bdng 3: PSNR cua cdc dnh HR tdi tao tic chuoi dnh Apaca

Chiing ta cd the thiy rang trong hiu het cac trudng hgp phuong phap KRI deu cho ket qua

tdt nhit

6.4 Thdi gian, dia diem va phirong phap nghien cuu

Hoat ddng nghien cuu chii yeu dugc thuc hien tai Bd mon Xu ly thdng tin va Phdng thi nghiem tich hgp Cac he thdng tich hgp thdng mmh (SIS) Dir lieu thuc nghiem (anh hien vi) dugc chup bang phan quang hpc cua thiet bi AFM tai Phdng thi nghiem Cdng nghe Micro-Nano De tai nghien cuu da tip trung giai quyet bai toan tren co sd ly thuyet ddng thdi su dung md phdng de kiem chung, Trong qua trinh thuc hien de tai cac kien thuc ly thuylt va kiln thuc thuc nghiem deu dugc chii trpng Cac ket qua nghien cuu da dugc viet thanh bao cao gui cho Hdi nghi khoa hpc chuyen nganh Viec hoan chinh mot nghien cuu vdi diy du co sd ly thuyet va phin mem md phdng tao dieu kien thuan Igi cho cdng viec giang day, nghien cuu khoa hpc cho sinh vien, hpc vien cao hpc ciing nhu tin dung de thuc hien cac de tai ve xu ly inh sau nay

6.5 Ket qua nghien ciru

6,5,L Cdc cong bo lien quan din kit qua ciia de tdi

Hong-Thinh Nguyen, Viet-Dung Nguyen, Ha Vu Le, "Super-Resolution Image

Construction from High-Speed Camera Sequences", Proceedings of the

International Conference on Advanced Technologies for Communications, Ho Chi

Minh City, October 2010

6,5,2, Kit qud dao tao cua De tdi

T)h tai da thu hiit dugc su tham gia cua cac sinh vien va hpc vien cao hpc trong khoa DTVT,

cd inh hudng tdt tdi Imh vuc dao tao Den nay da cd 2 sinh vien dai hpc va 2 hpc vien cao hpc tham gia nghien cuu de tai Hai sinh vien tham gia mpt phin nghien ciai cua de tai da dat giai nhat SV NCKH cip trudng va giai khuyen khich cip Bd GD-DT

Ket qua dao tao ciia de tai dugc tong kit trong cac bang dudi diy:

Danh muc cac khoa luan, luan van trong khudn kho dl tai QC.07.09

6,5,3, Ket qud irng dung ciia De tdi

Xiy dung mpt bd thu vien phin mem trong mdi trudng Matlab cho phep tao anh dp phin giai cao Phin mem da thii nghiem va hoat ddng tdt vdi cac loai anh chup bang may anh thudng va kinh hien vi dien tir; trong dd timg phin ciia thuat toan deu cd nhieu lira chpn phuang phap; dam bao cho ket qua tdt hon

6.6 Thao luan

Ket qua nhin dugc la dat so vdi cac yeu ciu dat ra ban diu Cac chuong truih don gian, nhanh va chay kha hieu qua

6.7 Ket luan va kien nghj

Trong tuong lai gin, viec tich hgp thuat toan vao cac thiet bi phin cung nhu DSP, FPGA se dugc thuc hien va hiia hen cii thien tdc dp xir ly; cd the trd thanh mpt san phim cd gia tri

Led cam <m

Cac thanh vien de tai xin chin thanh cam on tdi DHQGHN da tai trg cho de tai, tdi Phdng thi nghiem Cac he thdng tich hgp thdng minh (SIS), Phdng thi nghiem cdng nghe Micro-Nano da tao dieu kien cho phep sir dung toan bd cac trang thiet bi, tdi Trudng DHCN va Khoa DTVT ve sir quan tim va giiip dd hoan thanh de tai

6.8 Tai lieu tham khao

1 H Ur and D Gross, "Improved resolution from subpixel shifted pictures", CVGIP:

Graphical Models and Image Processing, Vol 54, No 2, March 1992, pp 181-186

2 A.M Tekalp, M.K Ozkan and M.I Sezan, "High-resolution image reconstruction from lower-resolution unage sequences and space-varying image restoration", in

Proceedings of the 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-92), Vol 3, 1992, pp 169-172

3 T Komatsu, T Igarashia, K Aizawab and T Saito, "Very high resolution miaging

scheme with multiple different-aperture cameras", Signal Processing: Image

Communication, Vol 5, No 5-6, December 1993, pp 511-526

4 R.C Hardie, K.J Barnard, J.G Bognar, E.E Armstrong and E.A Watson, resolution image reconstruction from a sequence of rotated and translated frames and

"High-its application to an infrared imaging system", Optical Engineering, Vol 37, No 1,

1998, pp 247-260

5 S Baker and T Kanade, "Super-resolution optical flow" Technical report, 1999

6 M Elad and A Feuer, "Super-resolution reconstruction of continuous image quences", in Proceedings of the 1999 Intemational Conference on Image Processing (ICIP-99), Vol 3, 1999, pp 459-463

se-7 N Nguyen and P Milanfar, "A wavelet-based interpolation-restoration method for superresolution", Circuits, Systems, and Signal Processing, Vol 19, No 4, 2000, pp 321-338

8 Y Altunbasak and A.J Patti, "A maximum a posteriori estimator for high resolution video reconstruction from MPEG video", in Proceedings of the 2000 Intemational Conference on Image Processing (ICIP-2000), Vol 2, 2000, pp 649-652

7 PHV LVC (cac giay tfr guri kern)

1 Trang bia cac khda luan va luan van

2 Ban sao cac bai bao, bao cao khoa hpc da cdng bd

3 Ban sao De cuang va Hgp ddng

4 Phieu tdm tat ket qua nghien cuu ciia de tai bang tieng Anh

5 Phieu dang ky ket qua nghien cuu KHCN

Ten cong trinh:

Thiet ke ki§n true cho bo loc thich nghi

SIP dung bien ddi song con

Ha Npi, thang 03 nam 2009