Chủ Nhật, 16 tháng 2, 2020

Practice for GRE #1

  • abate: subside or moderate - make sth less intense or severe
Steps are to be taken to abate pollution.
  • aberrant: abnormal or deviant
Aberrant behaviors of young boys should be seriously tracked and abated to prevent further crimes.
  • abeyance - suspended action
The plan to construct the new building is in abeyance because the governmental investment has not determined yet.
  • abscond - leave and hide secretly
Before the arrest decision a while, he has abscond with her girl friend.
  • abstemious - điều độ
Keeping an abstemious routine ameliorate my health conditions.
  • admonish: warn - sb for sth or for doing sth
Only by admonishing students to schedule their timetable did they save a gargantuan amount of time
  • adulterate - make sth impure by adding inferior substances  
She adulterates the fresh water source by saline.
  • aesthetic: artistic - connected with the beauty
The house is more aesthetic than functional
  • aggregate: gather 
The solutions should be aggregated to devise the best measure to attack the pollution.
  • alacrity: eagerness: great interest and excitement about something that is going to happen or about something that you want to do
I can not hide my happiness because of the alacrity about the new results.
  • alleviate: relieve
Morphin has been utilized to help him alleviate the pain.
  • amalgamate: combine
Sony and Nixon has been amalgamated into Nixy, the largest semiconductor manufacturer worldwide.
  • ambiguous: unclear in meaning
What he said about the incident is ambiguous. We can not reason anything further.
  • ambivalence: conflicting emotional attitudes
When she was leaving, I was in ambivalence towards our future.
  • ameliorate: improve
The amelioration of the manufacturing process can greatly benefit both companies and customers.
  • anachronism: misplaced in time
Many individuals argue that traditional customs are anachronism and should be disregarded.
  • analogous: comparable
The two courses are analogous in terms of syllabus.
  • anarchy: state of disorder
The period of anarchy in the governmental organization so far has no sign to be terminated.
  • anomalous: abnormal
The seemingly anomalous results in the experiments will be put into the discussion section today. 
  • antipathy: a strong feeling of dislike
I have an antipathy towards spicy foods because their negative effects on stomach.
  • apathy: lack of caring
The apathy from their parents is the main contributor to the anti-social behaviors today
  • pacify: appease - to bring peace or calm
Negotiation could be a viable approach to achieve pacification in this sensitive period.
  • apprise sb of sth: inform
To quickly apprise residents of emergencies can strongly mitigate the consequences.
  • approbation: approval
Being approbated by local authorities is the first step to succeed our program.
  • appropriate: acquire
Millions of dollar has been appropriated for AI research since the last year.
  • arduous: hard - involving a lot of effort and time
Despite the arduous situation, we will take a shot to to complete the project in time.
  • artless: without guile, open and honest
His artless expression persuades the council.
  • ascetic (adj or N): practicing self-denial
Being an ascetic somehow keep us away from the maelstrom of life.
  • assiduous: diligent: working hard
His assiduous efforts and attention to details are finally acknowledged.
  • assuage: soothe anger
The intense situation can be ameliorated by assuagement.  
  • attenuate: weaken, lessen
The signal is gradually attenuated on the cable over stages.
  • audacious: daring
His audacious decision surprises everyone in the class
  • austere: stern = really strict
Making educational program more austere could help students study more effectively.
  • aver: assert confidently or declar
The tangible consequences strongly aver the drawbacks of intensely smoking.
  • banal: common place,trite - a banal conversation
  • belie: contradict. to show a false impression of sth - belie sth
  • beneficent: being kind/helpful - a beneficent person
  • bolster: support - bolster sth - make sth stronger - improve sth
  • bombastic:pompous - sth with little meaning but to impress people - bombastic presentation/speakers
  • boorish: rude - people behavior 
  • burgeon - the burgeoning demand
  • burnish: polish - burnish sth
  • buttress: support (V) buttress sth
  • cacophonous: inharmonious 
  • capricious: unpredictable
  • castigation: punishment
  • catalyst: chất xúc tác
  • caustic: có tính chất ăn mòn - caustic comments/caustic liquid/chemical/cleaning product
  • chicanery: decption - sự lừa dối
  • congeal: solidify or coagulate, especially by cooling
  • coda: final passage of long-lasting thing
  • cogent: convincing - a cogent reason
  • commensurate: proportionate - sth to be commensurate with ST
  • compendious: brief - a compendious description
  • complaisant: willing to please others; obliging; agreeable - a complaisant person
  • complaint
  • reconciling: conciliatory (A): hòa giải  - reconcile sb to sth
  • condone: forgive - condone sth or condone doing sth
  • confound: confuse - confound sb
  • connoisseur: ​an expert on matters involving the judgement of beauty, quality or skill in art, food or music - an educational connoisseur
  • contention: claim - a belief or an opinion that you express, especially in an argument
  • contentious: quarrelsome: a contentious meeting ~ controversial
  • contrite: penitent: sám hối - an contrite action
  • conundrum: riddle
  • converge: approach
  • convoluted: intricate - complicated
  • cowardly: craven - lack of courage
  • daunt: intimidate: làm nản chí
  • decorum: good tastes in manner: đứng đắn
  • default: to fail to do sth - to default on STH
  • deference: humble submission and respect - in deference to sbd/sth
  • delineate: portray: vẽ phác họa - delineate sth
  • denigrate: blacken - làm tổn hại, phỉ báng ai đó - denigrate sbd/sth
  • deride: ridicule: chế nhạo ai đó
  • derivative: unoriginal - a derivative design/style
  • desiccate: dry up
  • desultory: aimless - a desultory conversation
  • deterrent (N): discourage - a deterrent to sbd/sth or deterrent against sth
  • diatribe (ˈdīəˌtrīb): bitter scolding: mắng nhiếc thậm tệ - diatribe against sth/sbd
  • dichotomy (dīˈkätəmē): split into two
  • diffidence: shyness (N)
  • diffuse: wordy - using too many of words or spread over a wide area - a diffuse writing style/a diffuse community
  • digression: wandering away from the subject: lạc đề - after several digression, he finally got to the point
  • dirge (N): lament with music - bài hát thê lương ở đám tang
  • disabuse: correct a false impression: disabuse sbd of sth
  • discerning: mentally quick and observant
  • discordant: conflicting - a discordant view
  • discredit: defame: mất uy tín
  • discrepancy: difference
  • discrete: separated
  • disingenuous: insincere - không chân thật
  • disjointed: lacking coherence
  • disparage: belittle: to suggest sbd/sth is not important or valuable - disparage sbd/sth
  • disparate: unrelated
  • dissemble: pretend
  • dissolution: looseness in morals: sth gradually disappears - dissolution of sth
  • dissonance: opposite in harmony: lack of agreement
  • distend: swell out: sưng lên
  • distill: purify

  • divest: deprive, strip - divest sb of sth. They are gonna divest him of the clothes
  • document: prove written evidence
  • dogmatic: opinionated - khăng khăng: a dogmatic approach
  • dormant: sleeping - a dormant volcano
  • dupe: easily fooled: dupe sbd in doing sth: He was duped into giving them his credit card
  • ebullient: showing excitement: he is in ebullient mood
  • eclectic: selective: different sources: he has very eclectic tastes in literature
  • efficacy: power to produce desired effect
  • effrontery: vô liêm sỉ: shameless boldness. This behavior maintains that he is effrontery
  • elegy: poem or song with lamentation: This elegy is for died people.
  • elicit: drawn out by discussion: i could elicit no response from him
  • embellish: enhance: the huge carved door was embellished with brass door knockers.
  • empirical: based on experience
  • emulate: imitate
  • endemic: prevailing amoung a group of people in an area: đặc hữu - an endemic tradition
  • enervate: weaken: enervate sbd - an enervating disease/climate
  • engender: cause (V) The issue engendered controversy
  • ephemeral: short-lived: ephemeral fashion/pleasures
  • equanimity: calmness of temperament: bình tĩnh
  • equivocate: lie: I am not sure! She equivocated
  • erudite: learned: scholarly: uyên bác - She was a well-respected and erudite scholar.  
  • esoteric: hard to understand - the esoteric texts
  • eulogy: expression of praise or funeral: a eulogy to marriage
  • euphemism: mild experssion: 
  • exacerbate: weaken
  • exculpate: clear from blame: 
  • exigency: urgent situation - He put financial exigency before personal sentiment
  • extrapolation: projection
  • facetious: joking - Stop being facetious; this is serious
  • fallacious: false: misleading - a fallacious argument
  • fatuous: brainless: a fatuous argument
  • fawning: trying to please by flattery: nịnh hót
  • felicitous: apt - a felicitous approach
  • fervor: a strong feeling about sth - She kissed him with unusual fervor
  • flag (V): droop: rủ xuống, tỏ vẻ yếu đuối
  • flout: reject - Motorists frequently flout the law
  • forestall: prevent taking action in advance
  • foment: stir up: instigate: xúi giục, kích động
  • gainsay: deny: nobody can gainsay his claims
  • garrulous: wordy - a garrulous old woman
  • goad: urge on: to keep annoying sb/st until they act - he goaded her into answering the question
  • grandiloquent: using long and complicated words to impress = pompous
  • gregarious: sociable: a gregarious man
  • gouge: overcharge - to force somebody to pay an unfairly high price for something
  • guileless: without deceit
  • harangue
  • iconoclastic: attacking cherished tradition
  • immutable: unchangeable
  • impair: hurt
  • impassive: without feeling - impassive expression or impassive face
  • imperturbable: calm: she faces challenges with imperturbable attitude
  • implacable: not capable of being pacified
  • implode: burst inwards - the windows on both sides of the house had imploded
  • inchorate: recently begun - The corona virus inchorates quickly
  • incongruity: lack of harmony - extremely silly and not logical and sensible
  • indigence: poverty
  • indolent: lazy: not wanting to work
  • inert: inactive
  • ingenuous: naive: innocent
  • inherent
  • insensible: unconscious
  • insipid: lacking flavor
  • insularity: narrow-minded, isolation
  • intractable: stubborn: unruly

  • loquacious: talkative
  • lucid: easy to understand - a lucid explanation
  • magnanimity: generosity
  • malinger: fake illness to escape duty
  • malleable: capable of being shaped
  • maverick: rebel - a maverick film director
  • mendacious: lying - mendacious press statements
  • misanthrope: a person who hates and avoids other people
  • mitigate: moderate
  • mollify: soothe: His explanation failed to mollify her
  • morose: sullen: ủ rũ, buồn rầu - she seems a bit morose today
  • neophyte: recent convert, beginner 
  • obdurate: stubborn - an obdurate attitude
  • obsequious: slavishly attentive: trying to hard to please sb - obsequious smiles

  • obviate: make unnecessary - This new evidence obviates any further inquiries
  • occlude: shut: close - to cover or block something
  • officious: meddlesome - kẻ lắm chuyện thích chúi mũi vào việc của ng khác
  • onerous: burdensome - an onerous duty/tax
  • opprobrium: vilification: infamy: sự sỉ nhục
  • partisan: committed to a party: one-sided
  • pathological: pertaining to disease - a pathological liar
  • paucity: scarcity: sự khan hiếm - the paucity of high-skilled labors
  • pedantic: showing off learning: bookish: mọt sách, nchuyen kiểu sách vở
  • penury: severe poverty
  • perennial: long-lasting

  • perfidious: disloyal: untrustworthy: treacherous - phản trắc, bất trung
  • perfunctory: superficial - hời hợt
  • permeable: porous: có nhiều lỗ hổng - permeable rocks
  • phlegmatic: calm - lãnh đạm
  • piety: devoutness: lòng thành kính, sự dâng hiến
  • plasticity: ability to be molded
  • platitude: trite remark: trivial statement
  • precipitate: rash: premature - his resignation precipitated a leadership crisis
  • prevaricate: lie - stop prevaricating and come to the point 
  • pristine: unspoiled - ​not developed or changed in any way; left in its original condition
  • probity: uprightness: thẳng thắn, cương trực

  • prodigal: wasteful - a prodigal administration
  • proliferate: grow rapidly
  • propitiate: appease: try to please someone
  • propriety: correct conduct: behaviors accepted by society
  • proscribe: banish: outlaw: officially say sth is banned
  • pungent: sharp in taste and smell - a pungent smell or burning rubber
  • quiescent: at rest: quite: not active
  • rarefied: made less dense: the rarefied atmosphere of academic life
  • recalcitrant: determined to resist authority: unwilling to obey rules or follow instructions; difficult to control - a recalcitrant child
  • recondite: profound: secret: not known about or understood by many people: a recondite subject: - thâm thúy, khó lường, sâu xa
  • refractory: stubborn

  • Amenable - easily persuaded

Because she is amenable, the plan is launched without any obstacles.
Being amenable is when you are easily persuade by someone.

  • Demur - to be reluctant
I got an acceptance in an US university last week but now I am demurring because many other chances come to me at a time.
The condition of being demurring is always unpleasant, but who know? The better you are, the more susceptible you become :(

  • Conceive (v) - conscious (adj) - consciousness (N)
  • innocuous - not harmful or offensive
  • ambivalent (adj) - ambivalence (N) to have both negative and positive feelings about sth
I have an ambivalent feeling about going abroad
I was really ambivalent when she told she loves me. Everything comes to fast

  • interminable = endless
  • harangue (N or V) = tirade = pompous speech
He harangues everyday about the partiality of the legal system.
After having an interminable harangue about his mother, Tom tells me that he needs to move out.


  • qualify = to limit (V)
He qualifies the invasion of the flood by a small dam. 
You interminable speech towards students should be qualified to encourage their enthusiasm.
  • maintain = to assert (V)
To mission of research is to maintain the hypotheses. 
Scientists are always looking for practical examples to maintain theory.
  • evasive adj - (lảng tráng) - not willing to give a clear answer to a question
Since he responded me in an evasive way, I totally gave up on him.
His action is evasive, which makes the boss become angry.

  • The wanting workforce in Japan urges the government to import cheap-price labors from South East Asian countries. 
  • bribe (N - countable) mua chuộc
She is very susceptible to bribes
  • susceptible (ajd) mẫn cảm, dễ bị kích thích
  • venality (N) the condition of being susceptible to bribes or corruption.
Even the most well-known sporting events are manipulated by the venality of officials to make biased decisions.

  • restive = restless (ajd) can not calm or rest
  • intimate (V)= to suggest something subtly
After a harangue to me tomorrow, I intimated me to leave for my future career path. I felt ambivalently since I was demurring to quit my job. The intimidation from Tom is a huge threat to a susceptible person like me, but I am not amenable finally.

  • parsimonious (adj) = extremely frugal = miserly (adj) = hà tiện/ khốn khổ
Everytime we ask him to go out for dinner, he refuses; and we know that he is miserly (parsimonious).

  • calumny (a): making a false statement to hurt another's reputation
Calumny is over the air and we should ignore and move on.
  • egregious (adj)~ notorious = standing out in a negative way, shockingly bad
The behaviors of the authority towards the disasters are egregious, causing the anger of crowd.
  • involved (adj)= complicated = the condition of being difficult to comprehend
Because of his pompous talk, the lecture becomes involved and nobody and perceive.
  • anomalous(adj) = not normal anomaly (N)
  • belie (V) to misrepresent
Despite an attempt to belie the incident, he have been inevitably arrested by cops.
  • prodigy (N) thần đồng
  • prodigal = rashly or wastefully extravagant = ngông cuồng/ ngạo mạn
The final winner are those who are not prodigal and always head forwards.


Thứ Ba, 11 tháng 2, 2020

Why does an internal covariate shift slow down the training procedure?

Let’s say you have a goal to reach, which is easier, a fixed goal vs a goal that keeps moving about? It is clear that a static goal is much easier to reach than a dynamic goal.
Each layer in a neural net has a simple goal, to model the input from the layer below it, so each layer tries to adapt to it’s input but for hidden layers, things get a bit complicated. The input’s statistical distribution changes after a few iterations, so if the input statistical distribution keeps changing, called internal covariate shift, the hidden layers will keep trying to adapt to that new distribution hence slowing down convergence. It is like a goal that keeps changing for hidden layers.
So the batch normalization (BN) algorithm tries to normalize the inputs to each hidden layer so that their distribution is fairly constant as training proceeds. This improves convergence of the neural net.

If a neuron is able to see the entire range of values across the entire training set that it is going to get as subsequent inputs, training can be made faster by normalizing the distribution (i.e. making its mean zero and variance one). The authors call it input whitening:
It has been long known (LeCun et al., 1998b; Wiesler & Ney, 2011) that the network training converges faster if its inputs are whitened – i.e., linearly transformed to have zero means and unit variances, and decorrelated. As each layer observes the inputs produced by the layers below, it would be advantageous to achieve the same whitening of the inputs of each layer. By whitening the inputs to each layer, we would take a step towards achieving the fixed distributions of inputs that would remove the ill effects of the internal covariate shift.

However, for a large training set, it is not computationally feasible to normalize inputs for each neuron. Hence, the normalization is carried out on a per-batch basis.

Batch Normalization may not always be optimal for learning

Batch normalization may lead to, say, inputs always lying in the linear range of sigmoid function. Hence the inputs are shifted and scaled by parameters γ and β. Optimal values of these hyperparameters can be learnt by using backpropagation.
Note that simply normalizing each input of a layer may change what the layer can represent. For instance, normalizing the inputs of a sigmoid would constrain them to the linear regime of the nonlinearity. To address this, we make sure that the transformation inserted in the network can represent the identity transform. To accomplish this, we introduce, for each activation x(k) , a pair of parameters γ(k), β(k), which scale and shift the normalized value:
y(k) = γ(k)xnormalized(k) + β(k).

So my curiosity is that is BN only used to speed up the training process? Does it improve the accuracy as I tested on a CNN training process, the performance even increases!