Updating network architecture
The connections between layers have associated “weights” that determine how much the output of one node contributes to the computation performed by the next, and training is a matter of adjusting those weights.
Input data is fed into the bottom layer, and the output of the top layer indicates the likelihood that the input fits into any of the available classes.
Before adaptation, the network with the conditional-random-field (CRF) layer slightly outperforms the one without (91.35% accuracy versus 91.06%).
The first transfer-learning method we examine is to simply expand the size of the trained network’s output layer and the layers immediately beneath it, to accommodate the addition of the new class. We then compare this approach to the one that uses the neural adapter.
Alexa scientists and engineers have poured a great deal of effort into Alexa’s core functionality, but through the Alexa Skills Kit, we’ve also enabled third-party developers to build their own Alexa skills — 70,000 and counting.
The type of adaptation — or “transfer learning” — that we study in the new paper would make it possible for third-party developers to make direct use of our in-house systems without requiring access to in-house training data.
Digital business demands speed, scale, and connectivity that yesterday’s network architecture simply can’t deliver.
Every aspect of the network architecture and infrastructure must support future growth and expansion without unnecessary upgrades.