--- references: - id: AicherAdaptively2020 accessed: - year: 2023 month: 4 day: 16 author: - family: Aicher given: Christopher - family: Foti given: Nicholas J. - family: Fox given: Emily B. citation-key: AicherAdaptively2020 container-title: Proceedings of The 35th Uncertainty in Artificial Intelligence Conference event-title: Uncertainty in Artificial Intelligence ISSN: 2640-3498 issued: - year: 2020 month: 8 day: 6 language: en page: 799-808 publisher: PMLR title: Adaptively Truncating Backpropagation Through Time to Control Gradient Bias type: paper-conference URL: https://proceedings.mlr.press/v115/aicher20a.html - id: Allen-ZhuCan2019 accessed: - year: 2019 month: 2 day: 10 author: - family: Allen-Zhu given: Zeyuan - family: Li given: Yuanzhi citation-key: Allen-ZhuCan2019 container-title: arXiv:1902.01028 [cs, math, stat] issued: - year: 2019 month: 2 day: 3 title: Can SGD Learn Recurrent Neural Networks with Provable Generalization? type: article-journal URL: http://arxiv.org/abs/1902.01028 - id: AndersonHighDimensional2017 author: - family: Anderson given: Alexander G. - family: Berg given: Cory P. citation-key: AndersonHighDimensional2017 container-title: arXiv:1705.07199 [cs] issued: - year: 2017 month: 5 day: 19 title: The High-Dimensional Geometry of Binary Neural Networks type: article-journal URL: http://arxiv.org/abs/1705.07199 - id: ArisoyDeep2012 accessed: - year: 2020 month: 5 day: 13 author: - family: Arisoy given: Ebru - family: Sainath given: Tara N. - family: Kingsbury given: Brian - family: Ramabhadran given: Bhuvana citation-key: ArisoyDeep2012 collection-title: WLM '12 container-title: >- Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT event-place: Montreal, Canada issued: - year: 2012 month: 6 day: 8 page: 20–28 publisher: Association for Computational Linguistics publisher-place: Montreal, Canada title: Deep neural network language models type: paper-conference - id: ArjovskyUnitary2016 author: - family: Arjovsky given: Martin - family: Shah given: Amar - family: Bengio given: Yoshua citation-key: ArjovskyUnitary2016 collection-title: ICML'16 container-title: >- Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 event-place: New York, NY, USA event-title: International Conference on Machine Learning issued: - year: 2016 month: 6 day: 11 language: en page: 1120-1128 publisher: JMLR.org publisher-place: New York, NY, USA title: Unitary Evolution Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1511.06464 - id: AuerLearning2008 accessed: - year: 2016 month: 6 day: 17 author: - family: Auer given: Peter - family: Burgsteiner given: Harald - family: Maass given: Wolfgang citation-key: AuerLearning2008 container-title: Neural Networks DOI: 10.1016/j.neunet.2007.12.036 issue: '5' issued: - year: 2008 page: 786–795 title: >- A learning rule for very simple universal approximators consisting of a single layer of perceptrons type: article-journal URL: http://www.igi.tugraz.at/maass/psfiles/126_web.pdf volume: '21' - id: BalduzziShattered2017 author: - family: Balduzzi given: David - family: Frean given: Marcus - family: Leary given: Lennox - family: Lewis given: J. P. - family: Ma given: Kurt Wan-Duo - family: McWilliams given: Brian citation-key: BalduzziShattered2017 container-title: PMLR event-title: International Conference on Machine Learning issued: - year: 2017 month: 7 day: 17 language: en page: 342-350 title: >- The Shattered Gradients Problem: If resnets are the answer, then what is the question? type: paper-conference URL: http://proceedings.mlr.press/v70/balduzzi17b.html - id: BazzaniRecurrent2017 author: - family: Bazzani given: Loris - family: Torresani given: Lorenzo - family: Larochelle given: Hugo citation-key: BazzaniRecurrent2017 issued: - year: 2017 language: en page: '15' title: Recurrent mixture density network for spatiotemporal visual attention type: article-journal - id: BengioLearning1994 author: - family: Bengio given: Y. - family: Simard given: P. - family: Frasconi given: P. citation-key: BengioLearning1994 container-title: IEEE Transactions on Neural Networks DOI: 10.1109/72.279181 ISSN: 1045-9227 issue: '2' issued: - year: 1994 month: 3 page: 157-166 title: Learning long-term dependencies with gradient descent is difficult type: article-journal URL: http://dsii.dsi.unifi.it/~paolo/ps/tnn-94-gradient.pdf volume: '5' - id: BengioScheduled2015 accessed: - year: 2017 month: 8 day: 6 author: - family: Bengio given: Samy - family: Vinyals given: Oriol - family: Jaitly given: Navdeep - family: Shazeer given: Noam citation-key: BengioScheduled2015 collection-title: NIPS'15 container-title: Advances in Neural Information Processing Systems 28 event-place: Cambridge, MA, USA issued: - year: 2015 page: 1171–1179 publisher: Curran Associates, Inc. publisher-place: Cambridge, MA, USA title: Scheduled sampling for sequence prediction with recurrent neural networks type: paper-conference URL: >- http://papers.nips.cc/paper/5956-scheduled-sampling-for-sequence-prediction-with-recurrent-neural-networks - id: BenTaiebBias2016 author: - family: Ben Taieb given: Souhaib - family: Atiya given: Amir F. citation-key: BenTaiebBias2016 container-title: IEEE transactions on neural networks and learning systems container-title-short: IEEE Trans Neural Netw Learn Syst DOI: 10.1109/TNNLS.2015.2411629 ISSN: 2162-2388 issue: '1' issued: - year: 2016 month: 1 language: eng page: 62-76 PMID: '25807572' title: A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting type: article-journal volume: '27' - id: Boulanger-LewandowskiModeling2012 accessed: - year: 2014 month: 5 day: 22 author: - family: Boulanger-Lewandowski given: Nicolas - family: Bengio given: Yoshua - family: Vincent given: Pascal citation-key: Boulanger-LewandowskiModeling2012 container-title: 29th International Conference on Machine Learning event-title: 29th International Conference on Machine Learning issued: - year: 2012 month: 6 day: 27 title: >- Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription type: paper-conference URL: http://arxiv.org/abs/1206.6392 - id: BownContinuousTime2006 accessed: - year: 2015 month: 1 day: 5 author: - family: Bown given: Oliver - family: Lexer given: Sebastian citation-key: BownContinuousTime2006 collection-number: '3907' collection-title: Lecture Notes in Computer Science container-title: Applications of Evolutionary Computing editor: - family: Rothlauf given: Franz - family: Branke given: Jürgen - family: Cagnoni given: Stefano - family: Costa given: Ernesto - family: Cotta given: Carlos - family: Drechsler given: Rolf - family: Lutton given: Evelyne - family: Machado given: Penousal - family: Moore given: Jason H. - family: Romero given: Juan - family: Smith given: George D. - family: Squillero given: Giovanni - family: Takagi given: Hideyuki ISBN: 978-3-540-33237-4 978-3-540-33238-1 issued: - year: 2006 month: 1 day: 1 language: en page: 652-663 publisher: Springer Berlin Heidelberg title: >- Continuous-Time Recurrent Neural Networks for Generative and Interactive Musical Performance type: chapter URL: http://link.springer.com/chapter/10.1007/11732242_62 - id: BuhusiWhat2005 accessed: - year: 2015 month: 6 day: 26 author: - family: Buhusi given: Catalin V. - family: Meck given: Warren H. citation-key: BuhusiWhat2005 container-title: Nature Reviews Neuroscience container-title-short: Nat Rev Neurosci DOI: 10.1038/nrn1764 ISSN: 1471-003X issue: '10' issued: - year: 2005 month: 10 language: en page: 755-765 title: What makes us tick? Functional and neural mechanisms of interval timing type: article-journal URL: >- http://www.researchgate.net/profile/Warren_Meck/publication/7600319_What_makes_us_tick_Functional_and_neural_mechanisms_of_interval_timing/links/02bfe50cd7b6077116000000.pdf volume: '6' - id: ChangAntisymmetricRNN2019 accessed: - year: 2019 month: 3 day: 20 author: - family: Chang given: Bo - family: Chen given: Minmin - family: Haber given: Eldad - family: Chi given: Ed H. citation-key: ChangAntisymmetricRNN2019 container-title: Proceedings of ICLR event-title: Seventh International Conference on Learning Representations issued: - year: 2019 month: 2 day: 25 title: 'AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks' type: paper-conference URL: http://arxiv.org/abs/1902.09689 - id: CharlesDistributed2016 author: - family: Charles given: Adam - family: Yin given: Dong - family: Rozell given: Christopher citation-key: CharlesDistributed2016 container-title: arXiv:1605.08346 [cs, math, stat] issued: - year: 2016 month: 5 day: 26 title: Distributed Sequence Memory of Multidimensional Inputs in Recurrent Networks type: article-journal URL: http://arxiv.org/abs/1605.08346 - id: ChevillonDirect2007 accessed: - year: 2018 month: 1 day: 13 author: - family: Chevillon given: Guillaume citation-key: ChevillonDirect2007 container-title: Journal of Economic Surveys DOI: 10.1111/j.1467-6419.2007.00518.x ISSN: 1467-6419 issue: '4' issued: - year: 2007 month: 9 day: 1 language: en page: 746-785 title: Direct Multi-Step Estimation and Forecasting type: article-journal URL: http://www.vwl.tuwien.ac.at/hanappi/AgeSo/rp/Chevillon_2007.pdf volume: '21' - id: ChoLearning2014 accessed: - year: 2016 month: 7 day: 13 author: - family: Cho given: Kyunghyun - family: Merrienboer given: Bart non-dropping-particle: van - family: Gulcehre given: Caglar - family: Bahdanau given: Dzmitry - family: Bougares given: Fethi - family: Schwenk given: Holger - family: Bengio given: Yoshua citation-key: ChoLearning2014 container-title: EMNLP 2014 event-title: EMNLP 2014 issued: - year: 2014 month: 6 day: 3 title: >- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation type: paper-conference URL: http://arxiv.org/abs/1406.1078 - id: ChoProperties2014 accessed: - year: 2016 month: 3 day: 30 author: - family: Cho given: Kyunghyun - family: Merriënboer given: Bart non-dropping-particle: van - family: Bahdanau given: Dzmitry - family: Bengio given: Yoshua citation-key: ChoProperties2014 container-title: arXiv preprint arXiv:1409.1259 issued: - year: 2014 title: 'On the properties of neural machine translation: Encoder-decoder approaches' type: article-journal URL: http://arxiv.org/abs/1409.1259 - id: ChungEmpirical2014 accessed: - year: 2016 month: 3 day: 30 author: - family: Chung given: Junyoung - family: Gulcehre given: Caglar - family: Cho given: KyungHyun - family: Bengio given: Yoshua citation-key: ChungEmpirical2014 container-title: NIPS issued: - year: 2014 month: 12 day: 11 title: Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling type: paper-conference URL: http://arxiv.org/abs/1412.3555 - id: ChungGated2015 accessed: - year: 2016 month: 3 day: 30 author: - family: Chung given: Junyoung - family: Gulcehre given: Caglar - family: Cho given: Kyunghyun - family: Bengio given: Yoshua citation-key: ChungGated2015 collection-title: ICML'15 container-title: >- Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37 event-title: International Conference on Machine Learning issued: - year: 2015 month: 2 day: 9 language: en page: 2067–2075 publisher: JMLR.org title: Gated Feedback Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1502.02367 - id: ChungHierarchical2016 author: - family: Chung given: Junyoung - family: Ahn given: Sungjin - family: Bengio given: Yoshua citation-key: ChungHierarchical2016 container-title: arXiv:1609.01704 [cs] issued: - year: 2016 month: 9 day: 6 title: Hierarchical Multiscale Recurrent Neural Networks type: article-journal URL: http://arxiv.org/abs/1609.01704 - id: ChungRecurrent2015 author: - family: Chung given: Junyoung - family: Kastner given: Kyle - family: Dinh given: Laurent - family: Goel given: Kratarth - family: Courville given: Aaron C - family: Bengio given: Yoshua citation-key: ChungRecurrent2015 container-title: Advances in Neural Information Processing Systems 28 editor: - family: Cortes given: C. - family: Lawrence given: N. D. - family: Lee given: D. D. - family: Sugiyama given: M. - family: Garnett given: R. issued: - year: 2015 page: 2980–2988 publisher: Curran Associates, Inc. title: A Recurrent Latent Variable Model for Sequential Data type: paper-conference URL: >- http://papers.nips.cc/paper/5653-a-recurrent-latent-variable-model-for-sequential-data.pdf - id: CollinsCapacity2016 author: - family: Collins given: Jasmine - family: Sohl-Dickstein given: Jascha - family: Sussillo given: David citation-key: CollinsCapacity2016 container-title: arXiv:1611.09913 [cs, stat] issued: - year: 2016 month: 11 day: 29 title: Capacity and Trainability in Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1611.09913 - id: CooijmansRecurrent2016 accessed: - year: 2017 month: 10 day: 6 author: - family: Cooijmans given: Tim - family: Ballas given: Nicolas - family: Laurent given: César - family: Gülçehre given: Çağlar - family: Courville given: Aaron citation-key: CooijmansRecurrent2016 container-title: arXiv preprint arXiv:1603.09025 issued: - year: 2016 title: Recurrent batch normalization type: article-journal URL: https://arxiv.org/abs/1603.09025 - id: DasguptaRegularized2016 accessed: - year: 2016 month: 10 day: 12 author: - family: Dasgupta given: Sakyasingha - family: Yoshizumi given: Takayuki - family: Osogami given: Takayuki citation-key: DasguptaRegularized2016 container-title: arXiv:1610.01989 [cs, stat] issued: - year: 2016 month: 9 day: 22 title: >- Regularized Dynamic Boltzmann Machine with Delay Pruning for Unsupervised Learning of Temporal Sequences type: article-journal URL: http://arxiv.org/abs/1610.01989 - id: DoellingCortical2015 accessed: - year: 2015 month: 11 day: 18 author: - family: Doelling given: Keith B. - family: Poeppel given: David citation-key: DoellingCortical2015 container-title: Proceedings of the National Academy of Sciences container-title-short: PNAS DOI: 10.1073/pnas.1508431112 ISSN: 0027-8424, 1091-6490 issue: '45' issued: - year: 2015 month: 10 day: 11 language: en page: E6233-E6242 PMID: '26504238' title: Cortical entrainment to music and its modulation by expertise type: article-journal URL: >- http://www.researchgate.net/publication/283293100_Cortical_entrainment_to_music_and_its_modulation_by_expertise?enrichId=rgreq-e38ec077-14ff-4ab1-9175-631623780941&enrichSource=Y292ZXJQYWdlOzI4MzI5MzEwMDtBUzoyOTA2NTc5OTk1NzI5OTNAMTQ0NjMwOTY3NTU0OA%3D%3D&el=1_x_2 volume: '112' - id: ElmanFinding1990 author: - family: Elman given: Jeffrey L citation-key: ElmanFinding1990 container-title: Cognitive Science DOI: 10.1016/0364-0213(90)90002-E issued: - year: 1990 page: 179-211 title: Finding structure in time type: article-journal volume: '14' - id: FortunatoBayesian2017 accessed: - year: 2018 month: 3 day: 21 author: - family: Fortunato given: Meire - family: Blundell given: Charles - family: Vinyals given: Oriol citation-key: FortunatoBayesian2017 container-title: arXiv:1704.02798 [cs, stat] issued: - year: 2017 month: 4 day: 10 title: Bayesian Recurrent Neural Networks type: article-journal URL: http://arxiv.org/abs/1704.02798 - id: FraccaroSequential2016 accessed: - year: 2016 month: 12 day: 7 author: - family: Fraccaro given: Marco - family: Sø nderby given: Sø ren Kaae - family: Paquet given: Ulrich - family: Winther given: Ole citation-key: FraccaroSequential2016 container-title: Advances in Neural Information Processing Systems 29 editor: - family: Lee given: D. D. - family: Sugiyama given: M. - family: Luxburg given: U. V. - family: Guyon given: I. - family: Garnett given: R. issued: - year: 2016 page: 2199–2207 publisher: Curran Associates, Inc. title: Sequential Neural Models with Stochastic Layers type: paper-conference URL: >- http://papers.nips.cc/paper/6039-sequential-neural-models-with-stochastic-layers.pdf - id: GalTheoretically2016 accessed: - year: 2017 month: 6 day: 21 author: - family: Gal given: Yarin - family: Ghahramani given: Zoubin citation-key: GalTheoretically2016 container-title: arXiv:1512.05287 [stat] event-title: NIPS issued: - year: 2016 title: A Theoretically Grounded Application of Dropout in Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1512.05287 - id: GersLearning2000 accessed: - year: 2016 month: 3 day: 30 author: - family: Gers given: Felix A. - family: Schmidhuber given: Jürgen - family: Cummins given: Fred citation-key: GersLearning2000 container-title: Neural Computation container-title-short: Neural Computation DOI: 10.1162/089976600300015015 ISSN: 0899-7667 issue: '10' issued: - year: 2000 month: 10 day: 1 page: 2451-2471 title: 'Learning to Forget: Continual Prediction with LSTM' type: article-journal volume: '12' - id: GersLearning2002 accessed: - year: 2017 month: 1 day: 19 author: - family: Gers given: Felix A. - family: Schraudolph given: Nicol N. - family: Schmidhuber given: Jürgen citation-key: GersLearning2002 container-title: Journal of machine learning research issue: Aug issued: - year: 2002 page: 115–143 title: Learning precise timing with LSTM recurrent networks type: article-journal URL: http://www.jmlr.org/papers/v3/gers02a.html volume: '3' - id: GilpinModel2023 accessed: - year: 2024 month: 1 day: 15 author: - family: Gilpin given: William citation-key: GilpinModel2023 container-title: Physical Review Research container-title-short: Phys. Rev. Res. DOI: 10.1103/PhysRevResearch.5.043252 issue: '4' issued: - year: 2023 month: 12 day: 15 page: '043252' publisher: American Physical Society title: >- Model scale versus domain knowledge in statistical forecasting of chaotic systems type: article-journal volume: '5' - id: GravesGenerating2013 accessed: - year: 2014 month: 11 day: 8 author: - family: Graves given: Alex citation-key: GravesGenerating2013 container-title: arXiv:1308.0850 [cs] issued: - year: 2013 month: 8 day: 4 title: Generating Sequences With Recurrent Neural Networks type: article-journal URL: http://arxiv.org/abs/1308.0850 - id: GravesPractical2011 accessed: - year: 2017 month: 9 day: 1 author: - family: Graves given: Alex citation-key: GravesPractical2011 collection-title: NIPS'11 container-title: >- Proceedings of the 24th International Conference on Neural Information Processing Systems event-place: USA ISBN: 978-1-61839-599-3 issued: - year: 2011 page: 2348–2356 publisher: Curran Associates Inc. publisher-place: USA title: Practical Variational Inference for Neural Networks type: paper-conference URL: >- https://papers.nips.cc/paper/4329-practical-variational-inference-for-neural-networks.pdf - id: GravesSupervised2012 author: - family: Graves given: Alex call-number: QA76.87 .G78 2012 citation-key: GravesSupervised2012 collection-number: v. 385 collection-title: Studies in computational intelligence event-place: Heidelberg ; New York ISBN: 978-3-642-24796-5 issued: - year: 2012 number-of-pages: '141' publisher: Springer publisher-place: Heidelberg ; New York title: Supervised sequence labelling with recurrent neural networks type: book URL: http://www.cs.toronto.edu/~graves/preprint.pdf - id: GregorDRAW2015 accessed: - year: 2015 month: 12 day: 8 author: - family: Gregor given: Karol - family: Danihelka given: Ivo - family: Graves given: Alex - family: Rezende given: Danilo Jimenez - family: Wierstra given: Daan citation-key: GregorDRAW2015 container-title: arXiv:1502.04623 [cs] issued: - year: 2015 month: 2 day: 16 title: 'DRAW: A Recurrent Neural Network For Image Generation' type: article-journal URL: http://arxiv.org/abs/1502.04623 - id: GruslysMemoryEfficient2016 author: - family: Gruslys given: Audrunas - family: Munos given: Remi - family: Danihelka given: Ivo - family: Lanctot given: Marc - family: Graves given: Alex citation-key: GruslysMemoryEfficient2016 container-title: Advances in Neural Information Processing Systems 29 editor: - family: Lee given: D. D. - family: Sugiyama given: M. - family: Luxburg given: U. V. - family: Guyon given: I. - family: Garnett given: R. issued: - year: 2016 page: 4125–4133 publisher: Curran Associates, Inc. title: Memory-Efficient Backpropagation Through Time type: paper-conference URL: >- http://papers.nips.cc/paper/6221-memory-efficient-backpropagation-through-time.pdf - id: GrzybWhich2009 author: - family: Grzyb given: B. J. - family: Chinellato given: E. - family: Wojcik given: G. M. - family: Kaminski given: W. A. citation-key: GrzybWhich2009 container-title: 2009 International Joint Conference on Neural Networks DOI: 10.1109/IJCNN.2009.5178822 event-title: 2009 International Joint Conference on Neural Networks issued: - year: 2009 month: 6 page: 1018-1024 title: Which model to use for the Liquid State Machine? type: paper-conference - id: GuCombining2021 accessed: - year: 2022 month: 8 day: 6 author: - family: Gu given: Albert - family: Johnson given: Isys - family: Goel given: Karan - family: Saab given: Khaled - family: Dao given: Tri - family: Rudra given: Atri - family: Ré given: Christopher citation-key: GuCombining2021 container-title: Advances in Neural Information Processing Systems DOI: 10.48550/arXiv.2110.13985 event-title: Advances in Neural Information Processing Systems issued: - year: 2021 page: 572–585 publisher: Curran Associates, Inc. title: >- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers type: paper-conference URL: http://arxiv.org/abs/2110.13985 volume: '34' - id: HardtGradient2018 author: - family: Hardt given: Moritz - family: Ma given: Tengyu - family: Recht given: Benjamin citation-key: HardtGradient2018 container-title: The Journal of Machine Learning Research container-title-short: J. Mach. Learn. Res. ISSN: 1532-4435 issue: '1' issued: - year: 2018 month: 1 day: 1 page: 1025–1068 title: Gradient descent learns linear dynamical systems type: article-journal URL: http://arxiv.org/abs/1609.05191 volume: '19' - id: HazanLearning2017 author: - family: Hazan given: Elad - family: Singh given: Karan - family: Zhang given: Cyril citation-key: HazanLearning2017 container-title: NIPS issued: - year: 2017 title: Learning Linear Dynamical Systems via Spectral Filtering type: paper-conference URL: http://arxiv.org/abs/1711.00946 - id: HazanTopological2012 accessed: - year: 2016 month: 6 day: 17 author: - family: Hazan given: Hananel - family: Manevitz given: Larry M. citation-key: HazanTopological2012 container-title: Expert Systems with Applications container-title-short: Expert Systems with Applications DOI: 10.1016/j.eswa.2011.06.052 ISSN: 0957-4174 issue: '2' issued: - year: 2012 month: 2 day: 1 page: 1597-1606 title: Topological constraints and robustness in liquid state machines type: article-journal URL: http://cs.haifa.ac.il/~manevitz/Publication/HazanManevitz.pdf volume: '39' - id: HePowerful2016 accessed: - year: 2016 month: 7 day: 6 author: - family: He given: Kun - family: Wang given: Yan - family: Hopcroft given: John citation-key: HePowerful2016 container-title: Advances in Neural Information Processing Systems event-title: NIPS issued: - year: 2016 month: 6 day: 15 title: >- A Powerful Generative Model Using Random Weights for the Deep Image Representation type: paper-conference URL: http://arxiv.org/abs/1606.04801 - id: HintonDeep2012 author: - family: Hinton given: G. - family: Deng given: Li - family: Yu given: Dong - family: Dahl given: G.E. - family: Mohamed given: A. - family: Jaitly given: N. - family: Senior given: A. - family: Vanhoucke given: V. - family: Nguyen given: P. - family: Sainath given: T.N. - family: Kingsbury given: B. citation-key: HintonDeep2012 container-title: IEEE Signal Processing Magazine DOI: 10.1109/MSP.2012.2205597 ISSN: 1053-5888 issue: '6' issued: - year: 2012 month: 11 page: 82-97 title: >- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups type: article-journal volume: '29' - id: HochreiterGradient2001 author: - family: Hochreiter given: Sepp - family: Bengio given: Yoshua - family: Frasconi given: Paolo - family: Schmidhuber given: Jürgen citation-key: HochreiterGradient2001 container-title: A field guide to dynamical recurrent neural networks issued: - year: 2001 publisher: IEEE Press title: >- Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies type: chapter URL: http://www.bioinf.jku.at/publications/older/ch7.pdf - id: HochreiterLong1997 accessed: - year: 2016 month: 3 day: 30 author: - family: Hochreiter given: Sepp - family: Schmidhuber given: Jürgen citation-key: HochreiterLong1997 container-title: Neural Computation container-title-short: Neural Computation DOI: 10.1162/neco.1997.9.8.1735 ISSN: 0899-7667 issue: '8' issued: - year: 1997 month: 11 day: 1 page: 1735-1780 title: Long Short-Term Memory type: article-journal URL: >- http://didawiki.di.unipi.it/lib/exe/fetch.php/magistraleinformatica/aa2/lstm.pdf volume: '9' - id: HochreiterLTSM1997 accessed: - year: 2017 month: 1 day: 19 author: - family: Hochreiter given: Sepp - family: Schmidhuber given: Jiirgen citation-key: HochreiterLTSM1997 container-title: >- Advances in Neural Information Processing Systems: Proceedings of the 1996 Conference issued: - year: 1997 page: 473–479 title: LTSM can solve hard time lag problems type: paper-conference URL: >- https://papers.nips.cc/paper/1215-lstm-can-solve-hard-long-time-lag-problems.pdf - id: HochreiterVanishing1998 accessed: - year: 2017 month: 7 day: 24 author: - family: Hochreiter given: Sepp citation-key: HochreiterVanishing1998 container-title: International Journal of Uncertainty Fuzziness and Knowledge Based Systems DOI: 10.1142/S0218488598000094 issued: - year: 1998 page: 107–115 title: >- The vanishing gradient problem during learning recurrent neural nets and problem solutions type: article-journal volume: '6' - id: HuszarHow2015 accessed: - year: 2018 month: 1 day: 13 author: - family: Huszár given: Ferenc citation-key: HuszarHow2015 container-title: arXiv:1511.05101 [cs, math, stat] issued: - year: 2015 month: 11 day: 16 title: >- How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? type: article-journal URL: http://arxiv.org/abs/1511.05101 - id: JaegerTutorial2002 accessed: - year: 2017 month: 10 day: 6 author: - family: Jaeger given: Herbert citation-key: JaegerTutorial2002 issued: - year: 2002 publisher: GMD-Forschungszentrum Informationstechnik title: >- Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the" echo state network" approach type: book URL: >- http://minds.jacobs-university.de/sites/default/files/uploads/papers/ESNTutorialRev.pdf volume: '5' - id: JingTunable2017 author: - family: Jing given: Li - family: Shen given: Yichen - family: Dubcek given: Tena - family: Peurifoy given: John - family: Skirlo given: Scott - family: LeCun given: Yann - family: Tegmark given: Max - family: Soljačić given: Marin citation-key: JingTunable2017 container-title: PMLR event-title: International Conference on Machine Learning issued: - year: 2017 month: 7 day: 17 language: en page: 1733-1741 title: >- Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs type: paper-conference URL: http://proceedings.mlr.press/v70/jing17a.html - id: JozefowiczEmpirical2015 accessed: - year: 2015 month: 12 day: 2 author: - family: Jozefowicz given: Rafal - family: Zaremba given: Wojciech - family: Sutskever given: Ilya citation-key: JozefowiczEmpirical2015 container-title: >- Proceedings of the 32nd International Conference on Machine Learning (ICML-15) issued: - year: 2015 page: 2342–2350 title: An empirical exploration of recurrent network architectures type: paper-conference URL: >- http://machinelearning.wustl.edu/mlpapers/paper_files/icml2015_jozefowicz15.pdf - id: KarpathyVisualizing2015 accessed: - year: 2015 month: 12 day: 2 author: - family: Karpathy given: Andrej - family: Johnson given: Justin - family: Fei-Fei given: Li citation-key: KarpathyVisualizing2015 container-title: arXiv:1506.02078 [cs] issued: - year: 2015 month: 6 day: 5 title: Visualizing and Understanding Recurrent Networks type: article-journal URL: http://arxiv.org/abs/1506.02078 - id: KatharopoulosTransformers2020 accessed: - year: 2020 month: 9 day: 16 author: - family: Katharopoulos given: Angelos - family: Vyas given: Apoorv - family: Pappas given: Nikolaos - family: Fleuret given: François citation-key: KatharopoulosTransformers2020 container-title: arXiv:2006.16236 [cs, stat] issued: - year: 2020 month: 8 day: 31 title: >- Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention type: article-journal URL: http://arxiv.org/abs/2006.16236 - id: KingmaImproving2016 author: - family: Kingma given: Diederik P. - family: Salimans given: Tim - family: Jozefowicz given: Rafal - family: Chen given: Xi - family: Sutskever given: Ilya - family: Welling given: Max citation-key: KingmaImproving2016 container-title: Advances in Neural Information Processing Systems 29 issued: - year: 2016 month: 6 day: 15 publisher: Curran Associates, Inc. title: Improving Variational Inference with Inverse Autoregressive Flow type: paper-conference URL: http://arxiv.org/abs/1606.04934 - id: KoutnikClockwork2014 accessed: - year: 2017 month: 6 day: 3 author: - family: Koutník given: Jan - family: Greff given: Klaus - family: Gomez given: Faustino - family: Schmidhuber given: Jürgen citation-key: KoutnikClockwork2014 container-title: arXiv:1402.3511 [cs] issued: - year: 2014 month: 2 day: 14 title: A Clockwork RNN type: article-journal URL: http://arxiv.org/abs/1402.3511 - id: KrishnamurthyTheory2022 author: - family: Krishnamurthy given: Kamesh - family: Can given: Tankut - family: Schwab given: David J. citation-key: KrishnamurthyTheory2022 container-title: Physical Review. X container-title-short: Phys Rev X DOI: 10.1103/physrevx.12.011011 ISSN: 2160-3308 issue: '1' issued: - year: 2022 language: eng page: '011011' PMCID: PMC9762509 PMID: '36545030' title: Theory of Gating in Recurrent Neural Networks type: article-journal URL: http://arxiv.org/abs/2007.14823 volume: '12' - id: KrishnanDeep2015 accessed: - year: 2017 month: 8 day: 11 author: - family: Krishnan given: Rahul G. - family: Shalit given: Uri - family: Sontag given: David citation-key: KrishnanDeep2015 container-title: arXiv preprint arXiv:1511.05121 issued: - year: 2015 title: Deep kalman filters type: article-journal URL: https://arxiv.org/abs/1511.05121 - id: LambProfessor2016 accessed: - year: 2018 month: 1 day: 13 author: - family: Lamb given: Alex - family: Goyal given: Anirudh - family: Zhang given: Ying - family: Zhang given: Saizheng - family: Courville given: Aaron - family: Bengio given: Yoshua citation-key: LambProfessor2016 container-title: Advances In Neural Information Processing Systems event-title: NIPS issued: - year: 2016 month: 10 day: 27 title: 'Professor Forcing: A New Algorithm for Training Recurrent Networks' type: paper-conference URL: http://arxiv.org/abs/1610.09038 - id: LaurentRecurrent2016 author: - family: Laurent given: Thomas - family: Brecht given: James non-dropping-particle: von citation-key: LaurentRecurrent2016 container-title: arXiv:1612.06212 [cs] issued: - year: 2016 month: 12 day: 19 title: A recurrent neural network without chaos type: article-journal URL: http://arxiv.org/abs/1612.06212 - id: LeCunGradientbased1998 author: - family: LeCun given: Y. citation-key: LeCunGradientbased1998 container-title: Proceedings of the IEEE DOI: 10.1109/5.726791 issue: '11' issued: - year: 1998 page: 2278-2324 title: Gradient-based learning applied to document recognition type: article-journal volume: '86' - id: LegensteinWhat2005 accessed: - year: 2016 month: 6 day: 17 author: - family: Legenstein given: Robert - family: Naeger given: Christian - family: Maass given: Wolfgang citation-key: LegensteinWhat2005 container-title: Neural Computation container-title-short: Neural Computation DOI: 10.1162/0899766054796888 ISSN: 0899-7667 issue: '11' issued: - year: 2005 month: 11 day: 1 page: 2337-2382 title: What Can a Neuron Learn with Spike-Timing-Dependent Plasticity? type: article-journal URL: http://www.igi.tu-graz.ac.at/maass/psfiles/154.pdf volume: '17' - id: LillicrapBackpropagation2019 accessed: - year: 2023 month: 4 day: 16 author: - family: Lillicrap given: Timothy P - family: Santoro given: Adam citation-key: LillicrapBackpropagation2019 collection-title: Machine Learning, Big Data, and Neuroscience container-title: Current Opinion in Neurobiology container-title-short: Current Opinion in Neurobiology DOI: 10.1016/j.conb.2019.01.011 ISSN: 0959-4388 issued: - year: 2019 month: 4 day: 1 language: en page: 82-89 title: Backpropagation through time and the brain type: article-journal volume: '55' - id: LiptonCritical2015 accessed: - year: 2015 month: 12 day: 2 author: - family: Lipton given: Zachary C. - family: Berkowitz given: John - family: Elkan given: Charles citation-key: LiptonCritical2015 container-title: arXiv:1506.00019 [cs] issued: - year: 2015 month: 5 day: 29 title: A Critical Review of Recurrent Neural Networks for Sequence Learning type: article-journal URL: http://arxiv.org/abs/1506.00019 - id: LukoseviciusReservoir2009 accessed: - year: 2016 month: 6 day: 17 author: - family: Lukoševičius given: Mantas - family: Jaeger given: Herbert citation-key: LukoseviciusReservoir2009 container-title: Computer Science Review container-title-short: Computer Science Review DOI: 10.1016/j.cosrev.2009.03.005 ISSN: 1574-0137 issue: '3' issued: - year: 2009 month: 8 page: 127-149 title: Reservoir computing approaches to recurrent neural network training type: article-journal URL: >- http://neuro.bstu.by/ai/To-dom/My_research/Papers-2.0/Echo-state-nn/2261_LukoseviciusJaeger09.pdf volume: '3' - id: MaassComputational2004 accessed: - year: 2016 month: 6 day: 17 author: - family: Maass given: W. - family: Natschläger given: T. - family: Markram given: H. citation-key: MaassComputational2004 container-title: 'Computational Neuroscience: A Comprehensive Approach' issued: - year: 2004 page: 575-605 publisher: Chapman & Hall/CRC title: Computational Models for Generic Cortical Microcircuits type: chapter URL: http://www.igi.tu-graz.ac.at/maass/psfiles/149-v05.pdf - id: MacKayReversible2018 accessed: - year: 2019 month: 1 day: 14 author: - family: MacKay given: Matthew - family: Vicol given: Paul - family: Ba given: Jimmy - family: Grosse given: Roger citation-key: MacKayReversible2018 container-title: Advances In Neural Information Processing Systems issued: - year: 2018 month: 10 day: 25 title: Reversible Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1810.10999 - id: MaddisonFiltering2017 accessed: - year: 2017 month: 8 day: 11 author: - family: Maddison given: Chris J. - family: Lawson given: Dieterich - family: Tucker given: George - family: Heess given: Nicolas - family: Norouzi given: Mohammad - family: Mnih given: Andriy - family: Doucet given: Arnaud - family: Teh given: Yee Whye citation-key: MaddisonFiltering2017 container-title: arXiv preprint arXiv:1705.09279 issued: - year: 2017 title: Filtering Variational Objectives type: article-journal URL: https://arxiv.org/abs/1705.09279 - id: MartensDeep2010 accessed: - year: 2018 month: 4 day: 2 author: - family: Martens given: James citation-key: MartensDeep2010 collection-title: ICML'10 container-title: >- Proceedings of the 27th International Conference on International Conference on Machine Learning event-place: USA ISBN: 978-1-60558-907-7 issued: - year: 2010 page: 735–742 publisher: Omnipress publisher-place: USA title: Deep Learning via Hessian-free Optimization type: paper-conference URL: http://www.cs.utoronto.ca/~jmartens/docs/Deep_HessianFree.pdf - id: MartensLearning2011 accessed: - year: 2018 month: 4 day: 2 author: - family: Martens given: James - family: Sutskever given: Ilya citation-key: MartensLearning2011 collection-title: ICML'11 container-title: >- Proceedings of the 28th International Conference on International Conference on Machine Learning event-place: USA ISBN: 978-1-4503-0619-5 issued: - year: 2011 page: 1033–1040 publisher: Omnipress publisher-place: USA title: Learning Recurrent Neural Networks with Hessian-free Optimization type: paper-conference URL: http://dl.acm.org/citation.cfm?id=3104482.3104612 - id: MartensTraining2012 author: - family: Martens given: James - family: Sutskever given: Ilya citation-key: MartensTraining2012 collection-title: Lecture Notes in Computer Science container-title: 'Neural networks: Tricks of the trade' ISBN: 978-3-642-35288-1 978-3-642-35289-8 issued: - year: 2012 page: 479–535 publisher: Springer title: Training deep and recurrent networks with Hessian-free optimization type: chapter URL: http://www.cs.toronto.edu/~jmartens/docs/HF_book_chapter.pdf - id: MhammediEfficient2017 author: - family: Mhammedi given: Zakaria - family: Hellicar given: Andrew - family: Rahman given: Ashfaqur - family: Bailey given: James citation-key: MhammediEfficient2017 container-title: PMLR event-title: International Conference on Machine Learning issued: - year: 2017 month: 7 day: 17 language: en page: 2401-2409 title: >- Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections type: paper-conference URL: http://proceedings.mlr.press/v70/mhammedi17a.html - id: MikolovRecurrent2010 author: - family: Mikolov given: Tomáš - family: Karafiát given: Martin - family: Burget given: Lukáš - family: Černockỳ given: Jan - family: Khudanpur given: Sanjeev citation-key: MikolovRecurrent2010 container-title: >- Eleventh Annual Conference of the International Speech Communication Association issued: - year: 2010 title: Recurrent Neural Network Based Language Model type: paper-conference URL: >- http://www.fit.vutbr.cz/research/groups/speech/servite/2010/rnnlm_mikolov.pdf - id: MillerWhen2018 accessed: - year: 2018 month: 8 day: 6 author: - family: Miller given: John - family: Hardt given: Moritz citation-key: MillerWhen2018 container-title: arXiv:1805.10369 [cs, stat] issued: - year: 2018 month: 5 day: 25 title: When Recurrent Models Don't Need To Be Recurrent type: article-journal URL: http://arxiv.org/abs/1805.10369 - id: MnihHumanlevel2015 author: - family: Mnih given: V. citation-key: MnihHumanlevel2015 container-title: Nature DOI: 10.1038/nature14236 issued: - year: 2015 page: 529-533 title: Human-level control through deep reinforcement learning type: article-journal volume: '518' - id: MohamedAcoustic2012 author: - family: Mohamed given: A. dropping-particle: r - family: Dahl given: G. E. - family: Hinton given: G. citation-key: MohamedAcoustic2012 container-title: IEEE Transactions on Audio, Speech, and Language Processing DOI: 10.1109/TASL.2011.2109382 ISSN: 1558-7916 issue: '1' issued: - year: 2012 month: 1 page: 14-22 title: Acoustic Modeling Using Deep Belief Networks type: article-journal volume: '20' - id: MonnerGeneralized2012 accessed: - year: 2016 month: 8 day: 11 author: - family: Monner given: Derek - family: Reggia given: James A. citation-key: MonnerGeneralized2012 container-title: Neural Networks container-title-short: Neural Networks DOI: 10.1016/j.neunet.2011.07.003 ISSN: 0893-6080 issued: - year: 2012 month: 1 page: 70-83 title: >- A generalized LSTM-like training algorithm for second-order recurrent neural networks type: article-journal URL: http://www.overcomplete.net/papers/nn2012.pdf volume: '25' - id: NeilPhased2016 accessed: - year: 2021 month: 9 day: 6 author: - family: Neil given: Daniel - family: Pfeiffer given: Michael - family: Liu given: Shih-Chii citation-key: NeilPhased2016 container-title: arXiv:1610.09513 [cs] issued: - year: 2016 month: 10 day: 29 title: >- Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences type: article-journal URL: http://arxiv.org/abs/1610.09513 - id: NiuRecurrent2019 accessed: - year: 2019 month: 6 day: 3 author: - family: Niu given: Murphy Yuezhen - family: Horesh given: Lior - family: Chuang given: Isaac citation-key: NiuRecurrent2019 container-title: arXiv:1904.12933 [quant-ph, stat] issued: - year: 2019 month: 4 day: 29 title: Recurrent Neural Networks in the Eye of Differential Equations type: article-journal URL: http://arxiv.org/abs/1904.12933 - id: Nussbaum-ThomAcoustic2016 author: - family: Nussbaum-Thom given: Markus - family: Cui given: Jia - family: Ramabhadran given: Bhuvana - family: Goel given: Vaibhava citation-key: Nussbaum-ThomAcoustic2016 DOI: 10.21437/Interspeech.2016-212 issued: - year: 2016 month: 9 day: 8 page: 390-394 title: Acoustic Modeling Using Bidirectional Gated Recurrent Convolutional Units type: paper-conference URL: http://www.isca-speech.org/archive/Interspeech_2016/pdfs/0212.PDF - id: OlivaStatistical2017 accessed: - year: 2017 month: 8 day: 12 author: - family: Oliva given: Junier B. - family: Poczos given: Barnabas - family: Schneider given: Jeff citation-key: OlivaStatistical2017 container-title: arXiv:1703.00381 [cs, stat] issued: - year: 2017 month: 3 day: 1 title: The Statistical Recurrent Unit type: article-journal URL: http://arxiv.org/abs/1703.00381 - id: PascanuDifficulty2013 author: - family: Pascanu given: Razvan - family: Mikolov given: Tomas - family: Bengio given: Yoshua citation-key: PascanuDifficulty2013 container-title: arXiv:1211.5063 [cs] event-title: Proceedings of The 30th International Conference on Machine Learning issued: - year: 2013 page: 1310-1318 title: On the difficulty of training Recurrent Neural Networks type: paper-conference URL: http://arxiv.org/abs/1211.5063 - id: PatrauceanSpatiotemporal2015 accessed: - year: 2016 month: 11 day: 1 author: - family: Patraucean given: Viorica - family: Handa given: Ankur - family: Cipolla given: Roberto citation-key: PatrauceanSpatiotemporal2015 container-title: arXiv:1511.06309 [cs] issued: - year: 2015 month: 11 day: 19 title: Spatio-temporal video autoencoder with differentiable memory type: article-journal URL: http://arxiv.org/abs/1511.06309 - id: PillonettoInterplay2016 author: - family: Pillonetto given: Gianluigi citation-key: PillonettoInterplay2016 container-title: arXiv:1612.09158 [cs, stat] issued: - year: 2016 month: 12 day: 29 title: The interplay between system identification and machine learning type: article-journal URL: http://arxiv.org/abs/1612.09158 - id: RavanbakhshDeep2016 author: - family: Ravanbakhsh given: Siamak - family: Schneider given: Jeff - family: Poczos given: Barnabas citation-key: RavanbakhshDeep2016 container-title: arXiv:1611.04500 [cs, stat] issued: - year: 2016 month: 11 day: 14 title: Deep Learning with Sets and Point Clouds type: paper-conference URL: http://arxiv.org/abs/1611.04500 - id: RobertsHierarchical2018 accessed: - year: 2018 month: 3 day: 21 author: - family: Roberts given: Adam - family: Engel given: Jesse - family: Raffel given: Colin - family: Hawthorne given: Curtis - family: Eck given: Douglas citation-key: RobertsHierarchical2018 container-title: arXiv:1803.05428 [cs, eess, stat] issued: - year: 2018 month: 3 day: 13 title: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music type: article-journal URL: http://arxiv.org/abs/1803.05428 - id: RohrbachLongShort2015 accessed: - year: 2015 month: 12 day: 2 author: - family: Rohrbach given: Anna - family: Rohrbach given: Marcus - family: Schiele given: Bernt citation-key: RohrbachLongShort2015 container-title: arXiv:1506.01698 [cs] issued: - year: 2015 month: 6 day: 4 title: The Long-Short Story of Movie Description type: article-journal URL: http://arxiv.org/abs/1506.01698 - id: RumelhartLearning1986 accessed: - year: 2018 month: 4 day: 3 author: - family: Rumelhart given: David E. - family: Hinton given: Geoffrey E. - family: Williams given: Ronald J. citation-key: RumelhartLearning1986 container-title: Nature DOI: 10.1038/323533a0 ISSN: 1476-4687 issue: '6088' issued: - year: 1986 month: 10 language: en page: 533-536 title: Learning representations by back-propagating errors type: article-journal URL: http://www.cs.toronto.edu/~hinton/absps/naturebp.pdf volume: '323' - id: RyderBlackbox2018 accessed: - year: 2018 month: 3 day: 21 author: - family: Ryder given: Thomas - family: Golightly given: Andrew - family: McGough given: A. Stephen - family: Prangle given: Dennis citation-key: RyderBlackbox2018 container-title: arXiv:1802.03335 [stat] issued: - year: 2018 month: 2 day: 9 title: Black-box Variational Inference for Stochastic Differential Equations type: article-journal URL: http://arxiv.org/abs/1802.03335 - id: SjobergNonlinear1995 accessed: - year: 2017 month: 9 day: 19 author: - family: Sjöberg given: Jonas - family: Zhang given: Qinghua - family: Ljung given: Lennart - family: Benveniste given: Albert - family: Delyon given: Bernard - family: Glorennec given: Pierre-Yves - family: Hjalmarsson given: Håkan - family: Juditsky given: Anatoli citation-key: SjobergNonlinear1995 collection-title: Trends in System Identification container-title: Automatica container-title-short: Automatica DOI: 10.1016/0005-1098(95)00120-8 ISSN: 0005-1098 issue: '12' issued: - year: 1995 month: 12 day: 1 page: 1691-1724 title: 'Nonlinear black-box modeling in system identification: a unified overview' type: article-journal URL: http://www.diva-portal.org/smash/get/diva2:315882/FULLTEXT02.pdf volume: '31' - id: SompolinskyChaos1988 accessed: - year: 2024 month: 1 day: 24 author: - family: Sompolinsky given: H. - family: Crisanti given: A. - family: Sommers given: H. J. citation-key: SompolinskyChaos1988 container-title: Physical Review Letters container-title-short: Phys. Rev. Lett. DOI: 10.1103/PhysRevLett.61.259 issue: '3' issued: - year: 1988 month: 7 day: 18 page: 259-262 publisher: American Physical Society title: Chaos in Random Neural Networks type: article-journal URL: https://link.aps.org/doi/10.1103/PhysRevLett.61.259 volume: '61' - id: SongNonlinear2020 accessed: - year: 2020 month: 2 day: 13 author: - family: Song given: Yang - family: Meng given: Chenlin - family: Liao given: Renjie - family: Ermon given: Stefano citation-key: SongNonlinear2020 container-title: arXiv:2002.03629 [cs, stat] issued: - year: 2020 month: 2 day: 10 language: en title: 'Nonlinear Equation Solving: A Faster Alternative to Feedforward Computation' type: article-journal URL: http://arxiv.org/abs/2002.03629 - id: SteilBackpropagationdecorrelation2004 author: - family: Steil given: J. J. citation-key: SteilBackpropagationdecorrelation2004 container-title: >- 2004 IEEE International Joint Conference on Neural Networks, 2004. Proceedings DOI: 10.1109/IJCNN.2004.1380039 event-title: >- 2004 IEEE International Joint Conference on Neural Networks, 2004. Proceedings issued: - year: 2004 month: 7 page: 843-848 vol.2 title: >- Backpropagation-decorrelation: online recurrent learning with O(N) complexity type: paper-conference URL: http://corwww.techfak.uni-bielefeld.de/system/files/Steil_BPDC_IJCNN2004.pdf volume: '2' - id: SuraceOnline2016 author: - family: Surace given: Simone Carlo - family: Pfister given: Jean-Pascal citation-key: SuraceOnline2016 issued: - year: 2016 title: >- Online Maximum Likelihood Estimation of the Parameters of Partially Observed Diffusion Processes type: paper-conference - id: SutskeverTraining2013 author: - family: Sutskever given: Ilya citation-key: SutskeverTraining2013 event-place: Toronto, Ont., Canada, Canada genre: PhD Thesis issued: - year: 2013 publisher: University of Toronto publisher-place: Toronto, Ont., Canada, Canada title: Training Recurrent Neural Networks type: thesis URL: https://tspace.library.utoronto.ca/handle/1807/36012 - id: TakamotoPDEBench2022 accessed: - year: 2022 month: 8 day: 4 author: - family: Takamoto given: Makoto - family: Praditia given: Timothy - family: Leiteritz given: Raphael - family: MacKinlay given: Dan - family: Alesiani given: Francesco - family: Pflüger given: Dirk - family: Niepert given: Mathias citation-key: TakamotoPDEBench2022 issued: - year: 2022 month: 6 day: 16 language: en title: 'PDEBench: An Extensive Benchmark for Scientific Machine Learning' type: paper-conference URL: https://openreview.net/forum?id=dh_MkX0QfrK - id: TallecUnbiasing2017 accessed: - year: 2023 month: 4 day: 16 author: - family: Tallec given: Corentin - family: Ollivier given: Yann citation-key: TallecUnbiasing2017 DOI: 10.48550/arXiv.1705.08209 issued: - year: 2017 month: 5 day: 23 number: arXiv:1705.08209 publisher: arXiv title: Unbiasing Truncated Backpropagation Through Time type: article URL: http://arxiv.org/abs/1705.08209 - id: TaylorModeling2006 accessed: - year: 2016 month: 6 day: 17 author: - family: Taylor given: Graham W. - family: Hinton given: Geoffrey E. - family: Roweis given: Sam T. citation-key: TaylorModeling2006 container-title: Advances in neural information processing systems issued: - year: 2006 page: 1345–1352 title: Modeling human motion using binary latent variables type: paper-conference URL: http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_693.pdf - id: TheisGenerative2015 accessed: - year: 2015 month: 12 day: 8 author: - family: Theis given: Lucas - family: Bethge given: Matthias citation-key: TheisGenerative2015 container-title: arXiv:1506.03478 [cs, stat] issued: - year: 2015 month: 6 day: 10 title: Generative Image Modeling Using Spatial LSTMs type: article-journal URL: http://arxiv.org/abs/1506.03478 - id: VisinReNet2015 accessed: - year: 2016 month: 3 day: 30 author: - family: Visin given: Francesco - family: Kastner given: Kyle - family: Cho given: Kyunghyun - family: Matteucci given: Matteo - family: Courville given: Aaron - family: Bengio given: Yoshua citation-key: VisinReNet2015 container-title: arXiv:1505.00393 [cs] issued: - year: 2015 month: 5 day: 3 title: >- ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks type: article-journal URL: http://arxiv.org/abs/1505.00393 - id: VoelkerLegendre author: - family: Voelker given: Aaron R - family: Kajic given: Ivana - family: Eliasmith given: Chris citation-key: VoelkerLegendre language: en page: '10' title: >- Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks type: article-journal - id: WangStateRegularized2019 accessed: - year: 2022 month: 6 day: 3 author: - family: Wang given: Cheng - family: Niepert given: Mathias citation-key: WangStateRegularized2019 DOI: 10.48550/arXiv.1901.08817 issued: - year: 2019 month: 5 day: 7 number: arXiv:1901.08817 publisher: arXiv title: State-Regularized Recurrent Neural Networks type: article URL: http://arxiv.org/abs/1901.08817 - id: WenMultiHorizon2017 accessed: - year: 2018 month: 1 day: 13 author: - family: Wen given: Ruofeng - family: Torkkola given: Kari - family: Narayanaswamy given: Balakrishnan citation-key: WenMultiHorizon2017 container-title: arXiv:1711.11053 [stat] issued: - year: 2017 month: 11 day: 29 title: A Multi-Horizon Quantile Recurrent Forecaster type: article-journal URL: http://arxiv.org/abs/1711.11053 - id: WerbosBackpropagation1990 author: - family: Werbos given: Paul J. citation-key: WerbosBackpropagation1990 container-title: Proceedings of the IEEE DOI: 10.1109/5.58337 ISSN: 0018-9219 issue: '10' issued: - year: 1990 month: 10 page: 1550-1560 title: 'Backpropagation through time: what it does and how to do it' type: article-journal URL: http://mail.werbos.com/Neural/BTT.pdf volume: '78' - id: WerbosGeneralization1988 accessed: - year: 2017 month: 9 day: 19 author: - family: Werbos given: Paul J. citation-key: WerbosGeneralization1988 container-title: Neural Networks container-title-short: Neural Networks DOI: 10.1016/0893-6080(88)90007-X ISSN: 0893-6080 issue: '4' issued: - year: 1988 month: 1 day: 1 page: 339-356 title: >- Generalization of backpropagation with application to a recurrent gas market model type: article-journal volume: '1' - id: WilliamsEfficient1990 accessed: - year: 2018 month: 4 day: 4 author: - family: Williams given: Ronald J. - family: Peng given: Jing citation-key: WilliamsEfficient1990 container-title: Neural Computation container-title-short: Neural Computation DOI: 10.1162/neco.1990.2.4.490 ISSN: 0899-7667 issue: '4' issued: - year: 1990 month: 12 day: 1 page: 490-501 title: >- An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories type: article-journal URL: >- https://www.researchgate.net/profile/Jing_Peng2/publication/2343555_An_Efficient_Gradient-Based_Algorithm_for_On-Line_Training_of_Recurrent_Network_Trajectories/links/555ca9a108ae8c0cab2a63ab/An-Efficient-Gradient-Based-Algorithm-for-On-Line-Training-of-Recurrent-Network-Trajectories.pdf volume: '2' - id: WilliamsLearning1989 accessed: - year: 2018 month: 1 day: 13 author: - family: Williams given: Ronald J. - family: Zipser given: David citation-key: WilliamsLearning1989 container-title: Neural Computation container-title-short: Neural Computation DOI: 10.1162/neco.1989.1.2.270 ISSN: 0899-7667 issue: '2' issued: - year: 1989 month: 6 day: 1 page: 270-280 title: A Learning Algorithm for Continually Running Fully Recurrent Neural Networks type: article-journal URL: >- https://pdfs.semanticscholar.org/8adb/8257a423f55b1f20ba62c8b20118d76a25c7.pdf volume: '1' - id: WisdomFullcapacity2016 accessed: - year: 2017 month: 4 day: 10 author: - family: Wisdom given: Scott - family: Powers given: Thomas - family: Hershey given: John - family: Le Roux given: Jonathan - family: Atlas given: Les citation-key: WisdomFullcapacity2016 container-title: Advances in Neural Information Processing Systems issued: - year: 2016 page: 4880–4888 title: Full-capacity unitary recurrent neural networks type: paper-conference URL: >- http://papers.nips.cc/paper/6327-full-capacity-unitary-recurrent-neural-networks - id: WisdomInterpretable2016 author: - family: Wisdom given: Scott - family: Powers given: Thomas - family: Pitton given: James - family: Atlas given: Les citation-key: WisdomInterpretable2016 container-title: Advances in Neural Information Processing Systems 29 issued: - year: 2016 month: 11 day: 22 title: Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery type: paper-conference URL: http://arxiv.org/abs/1611.07252 - id: WuMultiplicative2016 author: - family: Wu given: Yuhuai - family: Zhang given: Saizheng - family: Zhang given: Ying - family: Bengio given: Yoshua - family: Salakhutdinov given: Ruslan R citation-key: WuMultiplicative2016 container-title: Advances in Neural Information Processing Systems 29 editor: - family: Lee given: D. D. - family: Sugiyama given: M. - family: Luxburg given: U. V. - family: Guyon given: I. - family: Garnett given: R. event-title: NIPS issued: - year: 2016 page: 2856–2864 publisher: Curran Associates, Inc. title: On Multiplicative Integration with Recurrent Neural Networks type: paper-conference URL: >- http://papers.nips.cc/paper/6215-on-multiplicative-integration-with-recurrent-neural-networks.pdf - id: YaoDescribing2015 accessed: - year: 2015 month: 12 day: 2 author: - family: Yao given: Li - family: Torabi given: Atousa - family: Cho given: Kyunghyun - family: Ballas given: Nicolas - family: Pal given: Christopher - family: Larochelle given: Hugo - family: Courville given: Aaron citation-key: YaoDescribing2015 container-title: arXiv:1502.08029 [cs, stat] issued: - year: 2015 month: 2 day: 27 title: Describing Videos by Exploiting Temporal Structure type: article-journal URL: http://arxiv.org/abs/1502.08029 ...