Network parameters Part (i: First build input shape =( image size, image size, 1) batch size 128 the Encoder model Encoder/Decoder number of CNn layers and filters per layer layer filters =[32, 64 Original Information Build the autoencoder model first build the Encoder model inputs =Input(shape=input shape, name='encoder input) X= inputs Stack of cony2d blocks Notes: Encoder 1 )Use Batch Normalization before relu on deep networks 2)Use MaxPooling 2D as alternative to strides>1 # faster but not as good as strides>1 for filters in layer_ filters: x=Conv2D(filters=filters, kernel size=kernel size, strides=2 activation=relu Compressed Information Shape info needed to build Decoder Model shape =K int shape(x) Generate the latent vector x= Flatten((x) latent=Dense( latent dim, name='latent vector)x) Instantiate Encoder model encoder =Model(inputs, latent, name='encoderlp. Auto and variational encoders v. 9r6 encoder. summary
Part (ii):First build the Encoder Model • #part2 --------------------------------------------------- • # Network parameters • input_shape = (image_size, image_size, 1) • batch_size = 128 • kernel_size = 3 • latent_dim = 16 • # Encoder/Decoder number of CNN layers and filters per layer • layer_filters = [32, 64] • # Build the Autoencoder Model • # First build the Encoder Model • inputs = Input(shape=input_shape, name='encoder_input') • x = inputs • # Stack of Conv2D blocks • # Notes: • # 1) Use Batch Normalization before ReLU on deep networks • # 2) Use MaxPooling2D as alternative to strides>1 • # - faster but not as good as strides>1 • for filters in layer_filters: • x = Conv2D(filters=filters, • kernel_size=kernel_size, • strides=2, • activation='relu', • padding='same')(x) • # Shape info needed to build Decoder Model • shape = K.int_shape(x) • # Generate the latent vector • x = Flatten()(x) • latent = Dense(latent_dim, name='latent_vector')(x) • # Instantiate Encoder Model • encoder = Model(inputs, latent, name='encoder') • encoder.summary() Ch10. Auto and variational encoders v.9r6 16
#part3------- Part (ii): Build the Build the decoder model atent nut shut( latent di name-decoder nut) Decoder model x= Reshape((shape[ 1], shape [2], shape [3]))(x) Stack of Transposed Conv 2D blocks Notes: Compressed Information 1)Use Batch Normalization before Relu on deep networks 2)Use Up 2D as alternative to strides>1 #-faster but not as good as strides>1 for filters in layer filters[: -1] x= ConV2DTranspose(fil ters=fi lters, kernel size=kernel size strides=2, activation=relu, adding='same) Decoder x=Conv 2DTranspose(filters=1, kernel size=kernel size outputs= Activation(sigmoid, name='decoder output)(x) decoder= Model(latent inputs, outputs, name='decoder') decoder. summary Restored Information Autoencoder= Encoder+ Decoder autoencoder=Model(inputs, decoderfencoder(inputs)), name=autoencoder) autoencoder. summary Ch10. Auto and variational encoders v. 9r6 autoencoder. compile(loss='mse, optimizer'adam)
Part (iii):Build the Decoder Model • #part3 --------------------------------------------------- • # Build the Decoder Model • latent_inputs = Input(shape=(latent_dim,), name='decoder_input') • x = Dense(shape[1] * shape[2] * shape[3])(latent_inputs) • x = Reshape((shape[1], shape[2], shape[3]))(x) • # Stack of Transposed Conv2D blocks • # Notes: • # 1) Use Batch Normalization before ReLUon deep networks • # 2) Use UpSampling2D as alternative to strides>1 • # - faster but not as good as strides>1 • for filters in layer_filters[::-1]: • x = Conv2DTranspose(filters=filters, • kernel_size=kernel_size, • strides=2, • activation='relu', • padding='same')(x) • x = Conv2DTranspose(filters=1, • kernel_size=kernel_size, • padding='same')(x) • outputs = Activation('sigmoid', name='decoder_output')(x) • # Instantiate Decoder Model • decoder = Model(latent_inputs, outputs, name='decoder') • decoder.summary() • # Autoencoder = Encoder + Decoder • # Instantiate Autoencoder Model • autoencoder = Model(inputs, decoder(encoder(inputs)), name='autoencoder') • autoencoder.summary() • autoencoder.compile(loss='mse', optimizer='adam') Ch10. Auto and variational encoders v.9r6 17
#part4 train the autoencoder autoencoder. fit(x train noisy x train validation data=(x test noisy, x test), Part (iv Train the epochs=30, batch size=batch size) autoencoder Predict the autoencoder output from corrupted test images x decoded autoencoder predict(x test noisy) display the lst 8 corrupted and denoised images decode images ·rows,cos=10.30 num= rows co display result imgs =np concatenate(x test[ num], x test noisy[: numl x decoded[: numMI) imgs= imgs reshape(rows*3, cols, image size, image size) imgs=np vstack(np. split(imgs, rows, axis=1)) imgs imgs reshape((rows*3, -1, image size, image size)) imgs = np vstack([np hstack()for i in imgs) imgs =(imgs 255).astype np uint8) plt figure0 oIt title original images: top rows Corrupted Input: middle rows Denoised Input: third rows) plt. imshow( imgs, interpolation='none' cmap=gray) Image. fromarraylimgs) save( corrupted and denoised, anglers vgr6 plt showO
Part (iv): Train the autoencoder, decode images display result • #part4 --------------------------------------------------- • # Train the autoencoder • autoencoder.fit(x_train_noisy, • x_train, • validation_data=(x_test_noisy, x_test), • epochs=30, • batch_size=batch_size) • # Predict the Autoencoder output from corrupted test images • x_decoded = autoencoder.predict(x_test_noisy) • # Display the 1st 8 corrupted and denoised images • rows, cols = 10, 30 • num = rows * cols • imgs = np.concatenate([x_test[:num], x_test_noisy[:num], x_decoded[:num]]) • imgs = imgs.reshape((rows * 3, cols, image_size, image_size)) • imgs = np.vstack(np.split(imgs, rows, axis=1)) • imgs = imgs.reshape((rows * 3, -1, image_size, image_size)) • imgs = np.vstack([np.hstack(i) for i in imgs]) • imgs = (imgs * 255).astype(np.uint8) • plt.figure() • plt.axis('off') • plt.title('Original images: top rows, ' • 'Corrupted Input: middle rows, ' • 'Denoised Input: third rows') • plt.imshow(imgs, interpolation='none', cmap='gray') • Image.fromarray(imgs).save('corrupted_and_denoised.png') • plt.show() Ch10. Auto and variational encoders v.9r6 18
Codehttps://towardsdatascience.com/how-to-reduce-image-noises-by-autoencoder-65d5e6de543 Result: plt title original images: top rows, Corrupted Input middle rows, Denoised Input: third rows") 1“?4515084)9 503 3、+ 丶250350A4 70qc056 装速Q熟径 o70q056 157a 5“6S?854ψ10奂 0 器常亮索③经除猪称 备轻经关都染等 (0$"?;q 手巧新寒发网笔影公液图实托 804O sO028
Code https://towardsdatascience.com/how-to-reduce-image-noises-by-autoencoder-65d5e6de543 Result: plt.title('Original images: top rows, ' 'Corrupted Input: middle rows, ' 'Denoised Input: third rows') • '''Train s a d e no ising au toen cod er on MNIST datase t. • h ttp s://to ward sdatascie nce.co m/how-to-red uce -image -no ise s-by-au to enco de r-6 5d5 e6d e54 3 • De n o ising is one o f the classic ap plicatio ns of au toe ncod ers. • Th e d e n oisin g proce ss re move s u nwan te d noise that co rrupted th e • tru e sign al. • No ise + Data ---> Deno ising Autoe ncod er --> Data • Give n a train ing datase t of co rrup te d data as inp ut an d • tru e sign al as ou tput, a d eno ising au toe ncod er can re cove r the • h id d e n structure to ge ne rate clean d ata. • Th is e xamp le has mo dular de sign . Th e en cod er, d ecod er and au toe ncod er • are 3 mo d els that share we igh ts. Fo r e xamp le , after training the • au to e n co de r, the en cod er can be u sed to gen erate laten t ve ctors • o f in p ut data fo r low-dim visualizatio n like P CA o r TSNE. • ''' • # ke ras>> te nsorflow.keras, mod ification by kh w • fro m _ _ fu ture_ _ imp ort abso lu te _impo rt • fro m _ _ fu ture_ _ imp ort division • fro m _ _ fu ture_ _ imp ort print_fun ctio n • imp o rt te nso rflo w.ke ras as keras • fro m te n so rflo w.ke ras.laye rs imp ort Activation , Dense , Inpu t • fro m te n so rflo w.ke ras.laye rs imp ort Co nv2D, Flatte n • fro m te n so rflo w.ke ras.laye rs imp ort Re shap e, C onv2 DTran spose • fro m te n so rflo w.ke ras.mode ls impo rt Mo del • fro m te n so rflo w.ke ras imp ort backe nd as K • fro m te n so rflo w.ke ras.datase tsimpo rt mn ist • imp o rt n u mpy as np • imp o rt matp lo tlib.pyp lo tas plt • fro m P IL imp ort Image • n p .ran d o m.se ed(1 337 ) • # M NIST d atase t • (x_ train , _ ), (x_ te st, _) = mnist.load _d ata() • image _ size = x_train.shap e[1 ] • x_ train = n p.reshap e(x_ train , [-1, image _ size , image _size, 1 ]) • x_ te st = np .re shap e(x_te st, [-1 , image _size, image _ size , 1]) • x_ train = x_ train.astype ('float32 ') / 255 • x_ te st = x_ te st.astyp e('flo at32') / 2 55 • # Ge n e rate corru pted MNIST image s b y add in g noise with n ormal dist • # ce n te red at 0.5 an d std=0 .5 • n o ise = n p.rand om.normal(loc= 0.5, scale=0 .5 , size=x_ train.shap e) • x_ train _ n o isy = x_ train + n oise • n o ise = n p.rand om.normal(loc= 0.5, scale=0 .5 , size=x_ test.shap e) • x_ te st_ n oisy = x_te st+ no ise • x_ train _ n o isy = np .clip(x_ train _no isy, 0., 1 .) • x_ te st_ n oisy = np .clip(x_te st_ noisy, 0., 1.) • # Ne two rk parame ters • in p u t_ shap e = (image _ size , image _size, 1) • b atch _ size = 12 8 • ke rn e l_ size = 3 • late n t_ d im= 1 6 • # En co d e r/De cod er numb er of CNN laye rs an d filters pe r laye r • laye r_ filte rs = [32, 64] • # B u ild the Autoe ncod er Mod el • # First b u ild the En co der Mo del • in p u ts = Inpu t(sh ape =inpu t_shap e, name ='enco de r_inpu t') • x = in p uts • # Stack o f C o nv2 D blocks • # No te s: • # 1 ) Use B atch Normalization b efore R eLUo n dee p networks • # 2 ) Use M axP ooling2 D as alternative to strides> 1 • # - faste r b ut not as goo d as strid es>1 • fo r filte rs in laye r_filte rs: • x = C on v2D(filters=filte rs, • kern el_size=ke rne l_ size , • strid es=2 , • activation ='relu', • pad ding= 'same')(x) • # Sh ap e info ne ede d to b uild Deco de r M ode l • sh ap e = K.int_shap e(x) • # Ge n e rate the latent vecto r • x = Flatte n ()(x) • late n t = Dense (late nt_dim, name =' late nt_ve ctor')(x) • # In stan tiate En cod er Mod el • e n co d e r = M ode l(inp uts, laten t, name ='en code r') • e n co d e r.su mmary() • # B u ild the Deco der Mo del • late n t_ in puts = In put(shap e= (late nt_d im,), n ame= 'd eco der_ in put') • x = De n se(sh ape [1 ] * sh ape [2 ] * sh ape [3 ])(laten t_inpu ts) • x = R e sh ap e((shap e[1], sh ape [2 ], shap e[3]))(x) • # Stack o f Tran sp o sed C onv2 D blocks • # No te s: • # 1 ) Use B atch Normalization b efore R eLUo n dee p networks • # 2 ) Use UpSamp ling2 D as altern ative to stride s>1 • # - faste r b ut not as goo d as strid es>1 • fo r filte rs in laye r_filte rs[::-1]: • x = C on v2DTran spose (filters=filte rs, • kern el_size=ke rne l_ size , • strid es=2 , • activation ='relu', • pad ding= 'same')(x) • x = C o n v2DTran spo se(filters= 1, • ke rnel_size= kern el_size, • p add in g='same ')(x) • o u tp u ts = Activation('sigmoid', n ame =' deco der_ outp ut')(x) • # In stan tiate Deco der Mo del • d e co d e r = M ode l(laten t_inpu ts, outp uts, name = 'd ecod er') • d e co d e r.su mmary() • # Au to en code r = En co der + Decod er • # In stan tiate Autoen cod er Mod el • au to e n co de r = M ode l(inpu ts, de code r(enco der(inp uts)), name ='au to enco de r') • au to e n co de r.su mmary() • au to e n co de r.co mpile(lo ss='mse', o ptimize r='adam') • # Train th e au toe ncod er • au to e n co de r.fit(x_ train_n oisy, • x_train, • valid ation_ data= (x_test_n oisy, x_ test), • epo chs=3 0, • batch _size=b atch_ size ) • # P re d ict the Autoe nco der ou tp ut fro m corrup ted test image s • x_ d e co d ed = au toen cod er.pred ict(x_ te st_no isy) • # Disp lay th e 1st 8 co rrup te d and d eno ise d image s • ro ws, co ls = 1 0, 3 0 • num = ro ws * co ls • imgs = n p .co ncate nate([x_ te st[:nu m], x_ te st_no isy[:nu m], x_d eco ded [:nu m]]) • imgs = imgs.re sh ap e((rows * 3, co ls, image _size, image _ size ) • imgs = n p .vstack(n p.split(imgs, rows, axis= 1)) • imgs = imgs.re sh ap e((rows * 3, -1 , image _size, image _ size ) • imgs = n p .vstack([n p.hstack(i) fo r i in imgs]) • imgs = (imgs * 2 55).astyp e(np.uint8 ) • p lt.figu re () • p lt.axis('o ff') • p lt.title ('Original image s: top rows, ' • 'C orrup ted Inpu t: middle rows, ' • 'Den oised Inpu t: th ird ro ws') • p lt.imsh o w(imgs, interp olatio n='no ne', cmap = 'gray') • Image .fro marray(imgs).save ('co rru p te d_an d_ den oised.pn g') • p lt.sh o w() Ch10. Auto and variational encoders v.9r6 19
Exercise 3 Discuss applications of a vanilla( traditional) autoencoder. Ch10. Auto and variational encoders v.9r6
Exercise 3 • Discuss applications of a Vanilla (traditional) autoencoder. Ch10. Auto and variational encoders v.9r6 20