Skip to content
This repository has been archived by the owner on Dec 11, 2023. It is now read-only.

Added num_encoder_layers/num_decoder_layers to WMT16 standard hparams. #269

Closed
wants to merge 39 commits into from
Closed

Added num_encoder_layers/num_decoder_layers to WMT16 standard hparams. #269

wants to merge 39 commits into from

Conversation

tiberiu92
Copy link

This is based on the fix #265

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers
Corporate signers
  • Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
  • The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
  • The email used to register you as an authorized contributor must also be attached to your GitHub account.

@tiberiu92
Copy link
Author

I signed it !

@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

lmthang and others added 26 commits August 25, 2018 11:27
@lmthang
Add TrainOutputTuple, EvalOutputTuple, InferOutputTuple
8522665 
to allow for flexibility in extending models.
Clean up and factor train.py

PiperOrigin-RevId: 180703151
@lmthang
Wrap try/except for saver.restore and print variables in loaded check…
80ff1c9 
…points

PiperOrigin-RevId: 180960478
@lmthang
Improve several behaviors regarding loading hparams for train/inference:
653da78 
  (a) During inference, given --ckpt, we can try to load hparams in the same dir
  (b) When loading models and override_loaded_hparams=False, we still overwrite
      ["beam_width", "length_penalty_weight", "sampling_temperature",
       "num_translations_per_input"]
  (c) Introduce _add_argument to smartly add argument to hparams, so
      extend_hparams can be called when loading hparams.
      This is useful for old checkpoints.
  (d) Handle old checkpoints before the separation of num_layers into
      num_encoder_layers and num_decoder_layers.
Minor clean-ups of misc_utils.py.

PiperOrigin-RevId: 180989949
@lmthang
pretrained our models on
64e0436 
PiperOrigin-RevId: 181096467
@lmthang
Factor out model creation code in train.py and inference.py.
cfa4f48 
Update attention_model.py so that we can specify GNMT encoder without attention.

PiperOrigin-RevId: 181117151
@lmthang
Standardize vocab in test to use <unk>, <s>, </s>
989683d 
PiperOrigin-RevId: 181244462
@lmthang
Clean up inference_test.py and add more sharing code to _createTestIn…
656fb12 
…ferCheckpoint();

Rename ckpt into ckpt_path in inference.py and model_helper.py

PiperOrigin-RevId: 181260899
@eli7 @lmthang
NMT: Improving GPU availability debugging.
bdbba21 
op.device actually returns what the user requested not the actual device. This can be misleading as it can return "GPU0" even if no GPU is available.

For context see: tensorflow/tensorflow#1344

PiperOrigin-RevId: 181261953
@lmthang
Add add_info_summaries to automatically add summaries from info dict.
e73f83e 
Rename _get_best_results to get_best_results.
Update avg_grad_norm computation to divide by the number of examples instead.

PiperOrigin-RevId: 181346178
@lmthang
Internal change only
ae33cf1 
PiperOrigin-RevId: 181399302
@lmthang
Make compute_encoder_states() in model.py more general
4eba2d8 
PiperOrigin-RevId: 181765024
@lmthang
Set self.encoder_state_list in build_encoder() for gnmt_model.py
9be88ea 
PiperOrigin-RevId: 182319861
@lmthang
Add language_model flag to train a language model by ignoring the enc…
438e29a 
…oder.

PiperOrigin-RevId: 182426171
@oahziur @lmthang
Add infer_mode option to specify which type of decoder to use during …
3bb4930 
…inference.

PiperOrigin-RevId: 182960914
@lmthang
Replace compute_encoder_states with build_encoder_states (no sess.run).
eeed098 
Add an option include_embeddings to allow for
  appending embedding layer in front of encoder state list.
Properly handle the case when time_major=True.

PiperOrigin-RevId: 183117301
@oahziur @lmthang
Add sampled_softmax_loss and minor cleanup.
17c4272 
Useful when vocab size is very large.

PiperOrigin-RevId: 183184262
@oahziur @lmthang
Update standard hparams as num_layers is no longer a valid hparam.
4781773 
PiperOrigin-RevId: 183706548
@lmthang
Add copyright text for model_helper.py
005fef0 
PiperOrigin-RevId: 183778701
@eli7 @lmthang
Refactoring internal and external eval to allow injection of placehol…
0642c53 
…ders tensors.

PiperOrigin-RevId: 183781004
@eli7 @lmthang
Refactoring Model._build_encoder.
bd936dd 
- Allow the construction of encoders from sequences different from the default source sequence.
- Cleanups.

PiperOrigin-RevId: 184301964
@lmthang
Minor clean-ups, update docstring
781201a 
PiperOrigin-RevId: 184795279
@lmthang
Update vocab_utils.py to load embedding files under word2vec format.
3cd5d33 
Minor updates to nmt.py to print logging info on embedding files

PiperOrigin-RevId: 185313574
@oahziur @lmthang
Allow embedding file to have some misformated entry. Simply ignore th…
d2f66df 
…e entry

that doesn't have the correct size.

Handle attention_architecture == "" same as attention_architecture == "standard".

Use separate embedding partitioner for encoder and decoder.

PiperOrigin-RevId: 185489121
@lmthang
Unify reading and writing of hparams file.
6caa994 
PiperOrigin-RevId: 186098897
@oahziur @lmthang
Colocate output_layer with last LSTM cell to improve training speed.
8f20c69 
PiperOrigin-RevId: 186391226
@oahziur @lmthang
Add char-level embeddings for encoder only.
735b8b7 
PiperOrigin-RevId: 191382249
lmthang and others added 12 commits August 25, 2018 11:30
@lmthang
Refactored *model.py files.
a35b1e3 
Added a implicit flag extract_encoder_layers to get
  intermediate layers from GNMT models and skip decoder.

PiperOrigin-RevId: 191678516
@lmthang
Pretty print hparams when writing file.
6853265 
PiperOrigin-RevId: 191720585
@lmthang
Remove hparams.num_layers
2411c44 
PiperOrigin-RevId: 191804041
@oahziur @lmthang
Remove unused standard hparams.
9494594 
PiperOrigin-RevId: 196283435
@lmthang
Fix typo.
3128dac 
PiperOrigin-RevId: 203278814
@eli7 @lmthang
[NMT] Adding support for sharded train sets.
eb88c18 
PiperOrigin-RevId: 205470789
@lmthang
Decouple model loading from inference code
7932a59 
PiperOrigin-RevId: 207180389
@lmthang
Use LooseVersion for version check
471e484 
PiperOrigin-RevId: 207608855
@lmthang
Minor improvements and fixes
1355e32 
PiperOrigin-RevId: 208349749
@lmthang
Add INFERENCE_KEYS and make extend_hparams() in nmt.py more robust
b278487 
PiperOrigin-RevId: 210055078
@tiberiu92
hparams
b333eea
@tiberiu92
Added beam_search as default inference mode in hparams.
c045042
@tiberiu92 tiberiu92 closed this Oct 17, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@tiberiu92 @googlebot @tiberiuscarlat @lmthang @eli7 @oahziur