The Deep Learning Glossary
•Download as PPTX, PDF•
33 likes•26,828 views
The document is a glossary of deep learning terms from A-Z published by Re-Work, defining important concepts like artificial neural networks, convolutional neural networks, deep learning, embeddings, feedforward networks, generative adversarial networks, and more; each term includes a short definition and links to external sources for further explanation. The glossary aims to explain the key concepts underlying recent advances in deep learning that have enabled applications such as driverless cars, healthcare, and fashion recommendations.
More Related Content
What's hot
What's hot (20)
Similar to The Deep Learning Glossary
Similar to The Deep Learning Glossary (20)
More from NVIDIA
More from NVIDIA (20)
Recently uploaded
Recently uploaded (20)
The Deep Learning Glossary
- 2. DRIVERLESS CARS, BETTER PREVENTATIVE HEALTHCARE, AND EVEN BETTER FASHION RECOMMENDATIONS ARE ALL POSSIBLE TODAY BECAUSE OF DEEP LEARNING.
- 3. OUR FRIENDS AT RE-WORK PUBLISHED AN “A” TO “Z” DEEP LEARNING GLOSSARY. HERE ARE THE MOST IMPORTANT TERMS LINKED WITH RESOURCES FOR MORE IN-DEPTH EXPLORATION …
- 4. “PROCESSING DEVICES THAT ARE LOOSELY MODELLED AFTER THE NEURONAL STRUCTURE OF THE HUMAN BRAIN.” SOURCE: UNIVERSITY OF WISCONSIN-MADISON ARTIFICIAL NEURAL NETWORKS (ANN’S) NVIDIA Applied Research Compressing DNA Engine Facial Performance Video Classification
- 5. “DESCRIBES A LARGE VOLUME OF DATA – BOTH STRUCTURED AND UNSTRUCTURED – THAT INUNDATES A BUSINESS ON A DAY-TO-DAY BASIS.” SOURCE: SAS INSIGHTS BIG DATA AI-Accelerated Analytics For Industries Finance Telco IoT
- 6. “COMPRISED OF ONE OR MORE CONVOLUTIONAL LAYERS AND THEN FOLLOWED BY ONE OR MORE FULLY CONNECTED LAYERS AS IN A STANDARD MULTILAYER NEURAL NETWORK.” SOURCE: UFLDL CONVOLUTIONAL NEURAL NETWORKS NVIDIA Applied Research Detection & Classification of Hand Gestures Resource Efficient Inference
- 7. “FORM OF MACHINE LEARNING THAT ENABLES COMPUTERS TO LEARN FROM EXPERIENCE AND UNDERSTAND THE WORLD IN TERMS OF A HIERARCHY OF CONCEPTS.” SOURCE: GOODFELLOW, I., BENGIO, T., COURVILLE, A. DEEP LEARNING Deep Learning Applications Transforming Finance Detecting Cancer Deep Learning In Self-Driving Cars
- 8. “IS A REPRESENTATION OF INPUT, OR AN ENCODING. FOR EXAMPLE, A NEURAL WORD EMBEDDING IS A VECTOR THAT REPRESENTS THAT WORD.” SOURCE: DEEPLEARNING4J EMBEDDING Research Publication Generating An Embedded Neural Network
- 9. “ALLOW SIGNALS TO TRAVEL ONE WAY ONLY; FROM INPUT TO OUTPUT. THERE IS NO FEEDBACK (LOOPS) I.E. THE OUTPUT OF ANY LAYER DOES NOT AFFECT THAT SAME LAYER.” SOURCE: IMPERIAL COLLEGE FEEDFORWARD NETWORK Tutorial Creating a Feed-Forward Network With DIGITS
- 10. “TYPE OF AI ALGORITHMS USED IN UNSUPERVISED MACHINE LEARNING, IMPLEMENTED BY A SYSTEM OF TWO NEURAL NETWORKS COMPETING AGAINST EACH OTHER IN A ZERO-SUM GAME FRAMEWORK.” SOURCE: IAN GOODFELLOW GENERATIVE ADVERSARIAL NETWORKS Origin of Generative Adversarial Networks Ian Goodfellow Podcast GTC 17 Talk – Generative Adversarial Networks
- 11. “ARE AN ARCHITECTURE TO LET INFORMATION FLOW UNHINDERED ACROSS SEVERAL RNN LAYERS ON SO- CALLED “INFORMATION HIGHWAYS.” SOURCE: DEEPLEARNING4J HIGHWAY NETWORKS Research Publication Highway Networks for Visual Questions Answering
- 12. “TRAINING DEEP MODELS ARE SUFFICIENTLY DIFFICULT TASKS. MOST ALGORITHMS ARE STRONGLY AFFECTED BY THE CHOICE OF INITIALIZATION. THE INITIAL POINT CAN DETERMINE WHETHER THE ALGORITHM CONVERGES AT ALL, WITH SOME INITIAL POINTS BEING SO UNSTABLE THAT THE ALGORITHM ENCOUNTERS NUMERICAL DIFFICULTIES AND FAILS ALTOGETHER.” SOURCE: IAN GOODFELLOW INITIALIZATION Article Weight Initialization In Deep Neural Networks
- 13. “AN ARTIFICIAL NOISE ADDED TO THE INPUTS DURING TRAINING USED AS ANOTHER METHOD FOR REGULARIZATION AND IMPROVING GENERALIZATION OF A NEURAL NETWORK.” SOURCE: MICHAEL BRAGISNKEY, CTO AT AIDOC JITTER Tutorial What is Jitter?
- 14. “IS A TYPE OF UNSUPERVISED LEARNING, WHICH IS USED WHEN YOU HAVE UNLABELED DATA (I.E., DATA WITHOUT DEFINED CATEGORIES OR GROUPS). THE GOALS OF THIS ALGORITHM IS TO FIND GROUPS IN THE DATA, WITH THE NUMBER OF GROUPS REPRESENTED BY THE VARIABLE K.” SOURCE: DATA SCIENCE K-MEANS ALGORITHM Tutorial CUDA Implementation of the K-Means Clustering Algorithm
- 15. “FOR EACH PREDICTION, THERE IS AN ASSOCIATED NUMBER WHICH IS THE LOSS. FOR A TRUE PREDICTION, THE LOSS WILL BE SMALL AND FOR A TOTALLY WRONG PREDICTION THE LOSS WILL BE HIGH.” SOURCE: MICHAEL BRAGINSKY, CTO AT AIDOC LOSS FUNCTION NVIDIA Applied Research Loss Function for Image Restoration
- 16. “IS A FEEDFORWARD NEURAL NETWORK WITH MULTIPLE FULLY- CONNECTED LAYERS THAT USE NONLINEAR ACTIVATION FUNCTIONS TO DEAL WITH DATA WHICH IS NOT LINEARLY SEPARABLE. AN MLP IS THE MOST BASIC FORM OF A MULTILAYER NEURAL NETWORK, OR A DEEP NEURAL NETWORKS IF IT HAS MORE THAN 2 LAYERS.” SOURCE: DEEPLEARNING.NET MULTILAYER PERCEPTRON (MLP) Research Publication Multi-Layer Perceptron On A GPU
- 17. “IS THE COMPREHENSION BY COMPUTERS OF THE STRUCTURE AND MEANING OF HUMAN LANGUAGE (E.G., ENGLISH, SPANISH, JAPANESE), ALLOWING USERS TO INTERACT WITH THE COMPUTER USING NATURAL SENTENCES.” SOURCE: GARTNER RESEARCH NATURAL LANGUAGE PROCESSING Blog NVIDIA Developer – Natural Language Processing
- 18. “ONE-SHOT LEARNING IS WHEN AN ALGORITHM LEARNS FROM ONE OR A FEW NUMBER OF TRAINING EXAMPLES, CONTRAST TO THE TRADITIONAL MACHINE-LEARNING MODELS WHICH USES THOUSANDS EXAMPLES IN ORDER TO LEARN..” SOURCE: SUSHOVAN HALDAR ONE-SHOT LEARNING Research Publication One-Shot Imitation Learning with OpenAI & Berkeley
- 19. “TYPE OF LAYER COMMONLY FOUND IN CONVOLUTIONAL NEURAL NETWORKS, WHICH INTEGRATES INFORMATION FROM NEURONS WITH NEARBY RECEPTIVE FIELDS.” SOURCE: MICHAEL BRAGINSKY, CTA AT AIDOC POOLING Blog ParallelForall – Deep Learning In a Nutshell Core Concepts
- 20. “A NOVEL ARTIFICIAL AGENT, TERMED A DEEP Q-NETWORK, THAT CAN LEARN SUCCESSFUL POLICIES DIRECTLY FROM HIGH- DIMENSIONAL SENSORY INPUTS USING END-TO-END REINFORCEMENT LEARNING.” SOURCE: PETERSEN, S. (2015) HUMAN-LEVEL CONTROL THROUGH DEEP REINFORCEMENT LEARNING. Q-NETWORKS Blog Q-Network Trained to Play Breakout on OpenAI Gym
- 21. “A BRANCH OF MACHINE LEARNING THAT IS GOAL ORIENTATED; THAT IS, REINFORCEMENT LEARNING ALGORITHMS HAVE AS THEIR OBJECT TO MAXIMIZE A REWARD, OFTEN OVER THE COURSE OF MANY DECISIONS.” SOURCE: DEEPLEARNING4J REINFORCEMENT LEARNING Blog Deep Learning in a Nutshell: Reinforcement Learning
- 22. “IS A FUNCTION USED AS THE OUTPUT LAYER OF A NEURAL NETWORK THAT CLASSIFIES INPUT. IT CONVERTS VECTORS INTO CLASS PROBABILITIES. SOFTMAX NORMALIZES THE VECTOR OF SCORES BY FIRST EXPONENTIATING AND THEN DIVIDING BY A CONSTANT.” SOURCE: DEEPLEARNING4J SOFTMAX REGRESSION Tutorial Stanford – Softmax Regression
- 23. “ALLOWS US TO [TRAIN NEW MODELS] BY LEVERAGING THE ALREADY EXISTING LABELED DATA OF SOME RELATED TASK OR DOMAIN. WE TRY TO STORE THIS KNOWLEDGE GAINED IN SOLVING THE SOURCE TASK IN THE SOURCE DOMAIN AND APPLY IT TO OUR PROBLEM OF INTEREST.” SOURCE: SEBASTIAN RUDER TRANSFER LEARNING Research Publication Transfer Learning From Deep Features for Remote Sensing and Poverty Mapping
- 24. “IS A TYPE OF MACHINE LEARNING ALGORITHM USED TO DRAW INFERENCES FROM DATASETS CONSISTING OF INPUT DATA WITHOUT LABELED RESPONSES.” SOURCE: MATHWORKS UNSUPERVISED LEARNING Blog Using Unsupervised Learning For Artistic Style
- 25. “IS A DIRECTED MODEL THAT USES LEARNED APPROXIMATE INFERENCE AND CAN BE TRAINED PURELY WITH GRADIENT-BASED METHODS.” SOURCE: GOODFELLOW, IAN VARIATIONAL AUTOENCODER Blog Auto-Encoder Model Querying a Computer To Design Clothing
- 26. “TO PENALIZE LARGE WEIGHTS USING PENALTIES OR CONSTRAINTS ON THEIR SQUARED VALUES (L2 PENALTY) OR ABSOLUTE VALUES (L1 PENALTY).” SOURCE: HINTON, . G. NEURAL NETWORKS FOR MACHINE LEARNING WEIGHT DECAY Research Publication Weight Decay Can Improve Generalization
- 27. “THE PROCESS OF INITIALIZING WEIGHTS THAT THE VARIANCE REMAINS THE SAME FOR “X” AND “Y”. THIS INITIALIZATION PROCESS IS KNOWN AS XAVIER INITIALIZATION.” SOURCE: PRATEEK JOSHI XAVIER INITIALIZATION Research Publications Understanding the Difficulty of Training Deep Feedforward Neural Networks
- 28. “AS PIONEERS OF DEEP LEARNING, YOSHUA & YANN HAVE MADE SIGNIFICANT CONTRIBUTIONS TO THE ADVANCEMENTS OF THIS FIELD. THEY WILL BOTH BE JOINING THE DEEP LEARNING SUMMIT, MONTREAL IN OCTOBER, SPEAKING ON THE “PANEL OF PIONEERS.” SOURCE: RE-WORK GLOSSARY YOSHUA BENGIO & Their Work Advancing Robotics, Physics, and Medicine with AI – Yann LeCunn YANN LECUN The Rise of AI Through Deep Learning – Yoshua Bengio
- 29. “AN EXTREME FORM OF TRANSFER LEARNING, WHERE NO LABELED EXAMPLES ARE GIVEN AT ALL FOR THE ZERO-SHOT LEARNING TASK.” SOURCE: GOODFELLOW, IAN ZERO-SHOT LEARNING Research Publication Zero-Shot Learning Through Cross-Modal Transfer
- 30. Explore Now DISCOVER THE LATEST NEWS AND UPDATES ON DEEP LEARNING AND AI …
Editor's Notes
- Left justify “Embedded Computing”