tuning_GPT-2_with_own_text.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"02Train a GPT-2 Text-Generating Model w/ GPU","provenance":[{"file_id":"1VLG8e7YSEwypxU-noRNhsv5dW4NfTGce","timestamp":1610485290842},{"file_id":"1mMKGnVxirJnqDViH7BDJxFqWrsXlPSoK","timestamp":1555602712120}],"collapsed_sections":["HA7loTWXWwsg","ISWpRkEmXPVv","Bj2IJLHP3KwE","0wXB05bPDYxS","wmTXWNUygS5E"],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"H7LoMj4GA4n_"},"source":["#  Finetune a GPT-2 Text-Generating Model with own text\n","\n","based on  [Max Woolf](http://minimaxir.com) nootebook\n","\n","\n","\n"]},{"cell_type":"markdown","metadata":{"id":"HA7loTWXWwsg"},"source":["# 1. Installing GPT-2"]},{"cell_type":"code","metadata":{"id":"KBkpRgBCBS2_","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1611223007167,"user_tz":-60,"elapsed":11743,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"7dce321d-768c-4ae1-aabb-163701e85ce1"},"source":["%tensorflow_version 1.x\n","!pip install -q gpt-2-simple\n","import gpt_2_simple as gpt2\n","from datetime import datetime\n","# from google.colab import files"],"execution_count":1,"outputs":[{"output_type":"stream","text":["TensorFlow 1.x selected.\n","  Building wheel for gpt-2-simple (setup.py) ... \u001b[?25l\u001b[?25hdone\n","WARNING:tensorflow:\n","The TensorFlow contrib module will not be included in TensorFlow 2.0.\n","For more information, please see:\n","  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n","  * https://github.com/tensorflow/addons\n","  * https://github.com/tensorflow/io (for I/O related ops)\n","If you depend on functionality not listed there, please file an issue.\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"MX-wNTr8hjjE","executionInfo":{"status":"ok","timestamp":1611225516442,"user_tz":-60,"elapsed":546,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}}},"source":["from IPython.display import HTML\r\n","from IPython.display import Image"],"execution_count":23,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"eOn5bnZjW7JN"},"source":["# 2. Mounting google drive"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ZS0-z6XKPJT1","executionInfo":{"status":"ok","timestamp":1611223043401,"user_tz":-60,"elapsed":20096,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"68a48cdf-5eb5-47e3-c92f-f73125bd19c0"},"source":["#Mount Google Drive as folder\r\n","from google.colab import drive\r\n","drive.mount('/content/drive', force_remount=True)"],"execution_count":2,"outputs":[{"output_type":"stream","text":["Mounted at /content/drive\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"ISWpRkEmXPVv"},"source":["## Executing funcion to avoid disconnecting "]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":0},"id":"WtxVrSuuO5P1","executionInfo":{"status":"ok","timestamp":1611223055784,"user_tz":-60,"elapsed":1026,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"f01a8bb5-0e62-4326-df67-1da0d8ac9ce5"},"source":["import IPython\r\n","from google.colab import output\r\n","\r\n","display(IPython.display.Javascript('''\r\n"," function ClickConnect(){\r\n","   btn = document.querySelector(\"colab-connect-button\")\r\n","   if (btn != null){\r\n","     console.log(\"Click colab-connect-button\"); \r\n","     btn.click() \r\n","     }\r\n","   \r\n","   btn = document.getElementById('ok')\r\n","   if (btn != null){\r\n","     console.log(\"Click reconnect\"); \r\n","     btn.click() \r\n","     }\r\n","  }\r\n","  \r\n","setInterval(ClickConnect,60000)\r\n","'''))\r\n","\r\n","print(\"Done.\")"],"execution_count":3,"outputs":[{"output_type":"display_data","data":{"application/javascript":["\n"," function ClickConnect(){\n","   btn = document.querySelector(\"colab-connect-button\")\n","   if (btn != null){\n","     console.log(\"Click colab-connect-button\"); \n","     btn.click() \n","     }\n","   \n","   btn = document.getElementById('ok')\n","   if (btn != null){\n","     console.log(\"Click reconnect\"); \n","     btn.click() \n","     }\n","  }\n","  \n","setInterval(ClickConnect,60000)\n"],"text/plain":["<IPython.core.display.Javascript object>"]},"metadata":{"tags":[]}},{"output_type":"stream","text":["Done.\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"Bj2IJLHP3KwE"},"source":["## Verifying which GPU is active \n","\n","Colaboratory uses either a Nvidia T4 GPU or an Nvidia K80 GPU. The T4 is slightly faster than the old K80 for training GPT-2, and has more memory allowing you to train the larger GPT-2 models and generate more text.\n","\n","You can verify which GPU is active by running the cell below."]},{"cell_type":"code","metadata":{"id":"sUmTooTW3osf","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1611223061992,"user_tz":-60,"elapsed":735,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"3382a760-e104-4ff6-d826-792728550ceb"},"source":["!nvidia-smi"],"execution_count":4,"outputs":[{"output_type":"stream","text":["Thu Jan 21 09:57:44 2021       \n","+-----------------------------------------------------------------------------+\n","| NVIDIA-SMI 460.32.03    Driver Version: 418.67       CUDA Version: 10.1     |\n","|-------------------------------+----------------------+----------------------+\n","| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n","| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n","|                               |                      |               MIG M. |\n","|===============================+======================+======================|\n","|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n","| N/A   56C    P8    10W /  70W |      0MiB / 15079MiB |      0%      Default |\n","|                               |                      |                 ERR! |\n","+-------------------------------+----------------------+----------------------+\n","                                                                               \n","+-----------------------------------------------------------------------------+\n","| Processes:                                                                  |\n","|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n","|        ID   ID                                                   Usage      |\n","|=============================================================================|\n","|  No running processes found                                                 |\n","+-----------------------------------------------------------------------------+\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"0wXB05bPDYxS"},"source":["# 3. Downloading GPT-2 if you need to fine tune\n","\n","If you're retraining a model on new text, you need to download the GPT-2 model first. \n","\n","There are three released sizes of GPT-2:\n","\n","* `124M` (default): the \"small\" model, 500MB on disk.\n","* `355M`: the \"medium\" model, 1.5GB on disk.\n","* `774M`: the \"large\" model, cannot currently be finetuned with Colaboratory but can be used to generate text from the pretrained model (see later in Notebook)\n","* `1558M`: the \"extra large\", true model. Will not work if a K80 GPU is attached to the notebook. (like `774M`, it cannot be finetuned).\n","\n","Larger models have more knowledge, but take longer to finetune and longer to generate text. You can specify which base model to use by changing `model_name` in the cells below.\n","\n","The next cell downloads it from Google Cloud Storage and saves it in the Colaboratory VM at `/models/<model_name>`.\n","\n","This model isn't permanently saved in the Colaboratory VM; you'll have to redownload it if you want to retrain it at a later time."]},{"cell_type":"code","metadata":{"id":"P8wSlgXoDPCR","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1611223177115,"user_tz":-60,"elapsed":5948,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"238f2ea0-04e4-4fe9-d556-ddd7c71f5c75"},"source":["gpt2.download_gpt2(model_name=\"355M\")"],"execution_count":5,"outputs":[{"output_type":"stream","text":["Fetching checkpoint: 1.05Mit [00:00, 568Mit/s]                                                      \n","Fetching encoder.json: 1.05Mit [00:00, 144Mit/s]                                                    \n","Fetching hparams.json: 1.05Mit [00:00, 848Mit/s]                                                    \n","Fetching model.ckpt.data-00000-of-00001: 1.42Git [00:05, 270Mit/s]                                  \n","Fetching model.ckpt.index: 1.05Mit [00:00, 849Mit/s]                                                \n","Fetching model.ckpt.meta: 1.05Mit [00:00, 100Mit/s]                                                 \n","Fetching vocab.bpe: 1.05Mit [00:00, 174Mit/s]                                                       \n"],"name":"stderr"}]},{"cell_type":"markdown","metadata":{"id":"9fbdU8UVZ4_7"},"source":["## Txt file( for finetuning) below is a result of concatenation the couple of books, articles on the subject which is interesting for us (in this case ~5MB management leadership, good practices in business etc.)"]},{"cell_type":"code","metadata":{"id":"gp_4U3ScPsXX","executionInfo":{"status":"ok","timestamp":1611224518266,"user_tz":-60,"elapsed":596,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}}},"source":["file_name = \"business.txt\""],"execution_count":12,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"HeeSKtNWUedE"},"source":["If your text file is larger than 10MB, it is recommended to upload that file to Google Drive first, then copy that file from Google Drive to the Colaboratory VM."]},{"cell_type":"code","metadata":{"id":"-Z6okFD8VKtS","executionInfo":{"status":"ok","timestamp":1611224519714,"user_tz":-60,"elapsed":714,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}}},"source":["gpt2.copy_file_from_gdrive(file_name)"],"execution_count":13,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"LdpZQXknFNY3"},"source":["# 4. Finetune GPT-2\n","\n","The next cell will start the actual finetuning of GPT-2. It creates a persistent TensorFlow session which stores the training config, then runs the training for the specified number of `steps`. (to have the finetuning run indefinitely, set `steps = -1`)\n","\n","The model checkpoints will be saved in `/checkpoint/run1` by default. The checkpoints are saved every 500 steps (can be changed) and when the cell is stopped.\n","\n","The training might time out after 4ish hours; make sure you end training and save the results so you don't lose them!\n","\n","**IMPORTANT NOTE:** If you want to rerun this cell, **restart the VM first** (Runtime -> Restart Runtime). You will need to rerun imports but not recopy files.\n","\n","Other optional-but-helpful parameters for `gpt2.finetune`:\n","\n","\n","*  **`restore_from`**: Set to `fresh` to start training from the base GPT-2, or set to `latest` to restart training from an existing checkpoint.\n","* **`sample_every`**: Number of steps to print example output\n","* **`print_every`**: Number of steps to print training progress.\n","* **`learning_rate`**:  Learning rate for the training. (default `1e-4`, can lower to `1e-5` if you have <1MB input data)\n","*  **`run_name`**: subfolder within `checkpoint` to save the model. This is useful if you want to work with multiple models (will also need to specify  `run_name` when loading the model)\n","* **`overwrite`**: Set to `True` if you want to continue finetuning an existing model (w/ `restore_from='latest'`) without creating duplicate copies. "]},{"cell_type":"code","metadata":{"id":"aeXshJM-Cuaf"},"source":["sess = gpt2.start_tf_sess()\n","\n","gpt2.finetune(sess,\n","              dataset=file_name,\n","              model_name='\"355M\"',\n","              steps=5000,\n","              restore_from='fresh',\n","              run_name='\"355M\"',\n","              print_every=10,\n","              sample_every=200,\n","              save_every=500\n","              )"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"IXSuTNERaw6K"},"source":["After the model is trained, you can copy the checkpoint folder to your own Google Drive.\n","\n","If you want to download it to your personal computer, it's strongly recommended you copy it there first, then download from Google Drive. The checkpoint folder is copied as a `.rar` compressed file; you can download it and uncompress it locally."]},{"cell_type":"code","metadata":{"id":"VHdTL8NDbAh3"},"source":["gpt2.copy_checkpoint_to_gdrive(run_name='run355')"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"pel-uBULXO2L"},"source":["# 5. Load a Trained Model Checkpoint\n","\n","Running the next cell will copy the `.rar` checkpoint file from your Google Drive into the Colaboratory VM."]},{"cell_type":"code","metadata":{"id":"ncZGOOxSulvc","executionInfo":{"status":"ok","timestamp":1611224768181,"user_tz":-60,"elapsed":79616,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}}},"source":["cp /content/drive/MyDrive/GPT/checkpoint_run355.tar  /content/drive/MyDrive/"],"execution_count":14,"outputs":[]},{"cell_type":"code","metadata":{"id":"DCcx5u7sbPTD","executionInfo":{"status":"ok","timestamp":1611224838099,"user_tz":-60,"elapsed":67323,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}}},"source":["gpt2.copy_checkpoint_from_gdrive(run_name='run355')"],"execution_count":15,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"RTa6zf3e_9gV"},"source":["The next cell will allow you to load the retrained model checkpoint + metadata necessary to generate text.\n","\n","**IMPORTANT NOTE:** If you want to rerun this cell, **restart the VM first** (Runtime -> Restart Runtime). You will need to rerun imports but not recopy files."]},{"cell_type":"code","metadata":{"id":"KpgAl3nvzJW_"},"source":["#  !rm -R /content/checkpoint/run1/"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"zAgwTXHxzI8A","executionInfo":{"status":"ok","timestamp":1611045817946,"user_tz":-60,"elapsed":876,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"1b70a9e4-ab0b-45c4-fcf7-5a4f57d5e319"},"source":["cd /content"],"execution_count":null,"outputs":[{"output_type":"stream","text":["/content\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"s1cOoOg8y7Cb","executionInfo":{"status":"ok","timestamp":1611045819875,"user_tz":-60,"elapsed":902,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"d3a8a696-4cae-4759-812d-599c0c6130eb"},"source":["ls /content/checkpoint/run355/"],"execution_count":null,"outputs":[{"output_type":"stream","text":["checkpoint    events.out.tfevents.1611038631.2bacb2806925  model-5000.index\n","counter       hparams.json                                 model-5000.meta\n","encoder.json  model-5000.data-00000-of-00001               vocab.bpe\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"-fxL77nvAMAX","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1611224937682,"user_tz":-60,"elapsed":16025,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"552b0a6c-90f8-4ea3-f353-99cdf2849f83"},"source":["sess = gpt2.start_tf_sess()\n","gpt2.load_gpt2(sess, run_name='run355')"],"execution_count":17,"outputs":[{"output_type":"stream","text":["Loading checkpoint checkpoint/run355/model-5000\n","INFO:tensorflow:Restoring parameters from checkpoint/run355/model-5000\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"ClJwpF_ACONp"},"source":["# 6.Generate Text From The Trained Model\n","\n","After you've trained the model or loaded a retrained model from checkpoint, you can now generate text. `generate` generates a single text from the loaded model."]},{"cell_type":"code","metadata":{"id":"4RNY6RBI9LmL","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1611225268417,"user_tz":-60,"elapsed":27815,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"41244171-b7a6-408b-deb7-226af6df2626"},"source":["gpt2.generate(sess, run_name='run355')"],"execution_count":22,"outputs":[{"output_type":"stream","text":["fixed language and thinking patterns—both in the workplace and in life—can make or break a relationship. Emotional baggage—the things we have been taught in our education and from our parents—can get in the way of forging strong connections. And psychological well-being doesn’t stop at 30 days. We can linger beyond the end of a relationship or beyond a job.\n","\n","If you’ve suffered a setback or had a bad day at work, you may be struggling to rebuild your relationship with them. Your manager may have told you that you need to get better at keeping your emotions in check. The fight or flight response you feel may signal that you need to examine your emotional state more closely. If it turns out that your emotional state is unhealthy, you may feel justified in staying in that position.\n","\n","When we do engage in effective conflict management, we make sure we do it in a way that helps us grow. We make sure we use tools that help us learn and grow, and we make sure we practice using neutral language and thinking patterns that make it easy to engage in effective conflict resolution.\n","\n","For example, we learn to listen to our colleagues and then use our unique expertise to support and empower them. We work with them to help them solve complex problems, and we give them the space to be creative and develop their thinking. We also make sure we develop strong personal relationships with our team, and we give them autonomy to create their own path in their work. We see conflict resolution as a way to build both our professional and personal skills, and it works.\n","\n","Finally, we make sure we give people the time and space to be fully present in their work, which allows them to really connect emotionally. We create a sense of personal meaning and purpose in our work by being present in the moment, being in alignment with our colleagues, and being hands-on in all aspects of our work.\n","\n","\n","\n","\n","\n","How do I create a sense of personal meaning and purpose?\n","\n","\n","The first step is being aware of our overall emotional state and then actively managing that. Being aware of our overall emotional state and then actively managing it.\n","\n","We may not be able to change it completely, but we can ask questions to try and better understand what “wounds” we are dealing with and how they may be linked. We may need to work on becoming more present, aware of our overall emotional state, and then actively manage it.\n","\n","A good first step is to ask ourselves some questions to start:\n","\n","“What is keeping me feeling like I am not in sync with my colleagues?”\n","\n","“What is keeping me feeling like I am not in sync with my team?”\n","\n","“What is keeping me feeling like I am not in sync with my customers?”\n","\n","\n","\n","Once we understand where we are struggling with our overall emotional and physical health, we can begin to work on changing the environment in which we work. This can be as simple as limiting ourselves to a more manageable number of physical exams or having one-on-one meetings with our colleagues. Encouraging people to bring us together physically and spiritually through your everyday actions can help us to not only heal but build stronger relationships to come together with renewed purpose.\n","\n","A good friend or mentor can help us do this by enabling and encouraging physical and spiritual wellbeing. A physical or spiritual health and wellbeing coach can provide us with ideas on how to improve our overall health and well-being. A coach can also provide us with a set of simple actions to improve our overall wellbeing. Encouraging people to do something for us that we may not be able to do ourselves, or thinking positively about the things we’ve already tried, can help us to do better next time.\n","\n","Where could this help us achieve our potential?\n","\n","\n","\n","\n","\n","Chapter Eight\n","\n","\n","\n","\n","Improving Critical Thinking\n","\n","\n","\n","\n","\n","“Critical thinking is not a quick fix for all that. Sometimes, it’s the only way to survive.”\n","\n","Stephen Covey\n","\n","To be resilient mentally, we must remember that we are all here for just a few moments in our lives. There is a span of time when we will all be here, doing things that really matter and that have bearing on our wellbeing.\n","\n","When this happens, and you have made it to this point, you have achieved what you set out to do. You have made it clear, in no uncertain terms, who you are and what you want. And in the context of this book, that is really what this book is all about. You have achieved your primary objectives. You have clearly defined the major themes that you plan to tackle. You have given your team a vision and purpose. You have outlined the major tasks that you expect each member of your team to do. Now, to deliver, you need to get the team moving forwards and not backwards. You need clarity on when you will need to\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"oF4-PqF0Fl7R"},"source":["If you're creating an API based on your model and need to pass the generated text elsewhere, you can do `text = gpt2.generate(sess, return_as_list=True)[0]`\n","\n","You can also pass in a `prefix` to the generate function to force the text to start with a given character sequence and generate text from there (good if you add an indicator when the text starts).\n","\n","You can also generate multiple texts at a time by specifing `nsamples`. Unique to GPT-2, you can pass a `batch_size` to generate multiple samples in parallel, giving a massive speedup (in Colaboratory, set a maximum of 20 for `batch_size`).\n","\n","Other optional-but-helpful parameters for `gpt2.generate` and friends:\n","\n","*  **`length`**: Number of tokens to generate (default 1023, the maximum)\n","* **`temperature`**: The higher the temperature, the crazier the text (default 0.7, recommended to keep between 0.7 and 1.0)\n","* **`top_k`**: Limits the generated guesses to the top *k* guesses (default 0 which disables the behavior; if the generated output is super crazy, you may want to set `top_k=40`)\n","* **`top_p`**: Nucleus sampling: limits the generated guesses to a cumulative probability. (gets good results on a dataset with `top_p=0.9`)\n","* **`truncate`**: Truncates the input text until a given sequence, excluding that sequence (e.g. if `truncate='<|endoftext|>'`, the returned text will include everything before the first `<|endoftext|>`). It may be useful to combine this with a smaller `length` if the input texts are short.\n","*  **`include_prefix`**: If using `truncate` and `include_prefix=False`, the specified `prefix` will not be included in the returned text."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":[]},{"cell_type":"markdown","metadata":{"id":"Leev036QhESP"},"source":["# 7. Generate some text with given start sentence/prefix"]},{"cell_type":"code","metadata":{"id":"86DVgGBNhpKk"},"source":[],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"cfgCZxtJtdCz","executionInfo":{"status":"ok","timestamp":1611229609460,"user_tz":-60,"elapsed":25777,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"cf006b39-ea30-4324-d2a2-4ad6a431976d"},"source":["#@title Prefix text { display-mode: \"form\" }\r\n","give_run_name = 'run355' #@param [\"None\"] {type:\"raw\", allow-input: true}\r\n","paste_prefix_text = \"Modern  leader knows what to do.\" #@param [\"\"] {allow-input: true}\r\n","set_temperature = 0.5 #@param {type:\"slider\", min:0.1, max:1, step:0.05}\r\n","set_text_length = 420 #@param {type:\"slider\", min:0, max:1023, step:10}\r\n","set_nr_samples = 5 #@param {type:\"slider\", min:4, max:10, step:1}\r\n","\r\n","\r\n","\r\n","print(\"please wait text is generated...\" ) \r\n","\r\n","gpt2.generate(sess,\r\n","              run_name=give_run_name,\r\n","              length=set_text_length,\r\n","              temperature=set_temperature,\r\n","              prefix=paste_prefix_text,\r\n","              nsamples=set_nr_samples,\r\n","              batch_size=set_nr_samples\r\n","              )"],"execution_count":80,"outputs":[{"output_type":"stream","text":["please wait text is generated...\n","Modern  leader knows what to do.\n","\n","\n","\n","\n","\n","3. Collaborate and Build Team\n","\n","Leaders who want to lead effective teams must be able to get along with people who are not as experienced or as smart as they are. That means making it a point to spend time with and work with people who are like themselves, and not judging them because they have different backgrounds.\n","\n","In developing a team, leaders need to be careful not to make everyone feel like a member of the team. That can lead to resentment and division. Instead, create opportunities for team members to bond and develop their own ideas. That can help employees share ideas and differentiate themselves, and it will lead to more collaboration down the road.\n","\n","For example, let’s say your boss expects a team member to be efficient and effective at all times. You could, for example, give her some tasks that she is not suited for and ask her to work on something that she would like to do but cannot due to a business need. She might be able to do this for a few hours, then decide that she can’t do that for another few weeks. That would be a good result for the team.\n","\n","Of course, you cannot treat everyone equally. It is the responsibility of the leader to ensure that all members of the team are doing their jobs with professionalism and respect. If that is not the case, then the leader must make sure that the team has opportunities to grow and learn, both on and off the clock. If that does not happen, then it is the responsibility of the team to ensure that the individual is in line with the rest of the team.\n","\n","If you have a high level of trust within the team, then it will help to give the team members the opportunity to develop their own ideas and ask questions of the leader. The team might not get the chance to grow big and strong, but it will continue to learn and improve. And if the team members do not develop their own ideas and ask questions of the leader, then the leader will be held accountable if the ideas they come\n","====================\n","Modern  leader knows what to do.\n","\n","\n","\n","\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n","====================\n","Modern  leader knows what to do.\n","\n","\n","\n","\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n","====================\n","Modern  leader knows what to do.\n","\n","\n","\n","\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n","====================\n","Modern  leader knows what to do.\n","\n","\n","\n","\n","\n","THE ENABLING PLAY\n","\n","\n","\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n","====================\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"tzRHMT1jynjj"},"source":[],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"VGbkorN8yoEy","executionInfo":{"status":"ok","timestamp":1611230048419,"user_tz":-60,"elapsed":26068,"user":{"displayName":"Marek Leszczynski","photoUrl":"","userId":"05664549655810509768"}},"outputId":"1ae1c479-fc07-4367-e1c6-dacffb29abfd"},"source":["#@title Prefix text { display-mode: \"form\" }\r\n","give_run_name = 'run355' #@param [\"None\"] {type:\"raw\", allow-input: true}\r\n","paste_prefix_text = \"When you don't know what to do.\" #@param [\"\"] {allow-input: true}\r\n","set_temperature = 0.5 #@param {type:\"slider\", min:0.1, max:1, step:0.05}\r\n","set_text_length = 420 #@param {type:\"slider\", min:0, max:1023, step:10}\r\n","set_nr_samples = 5 #@param {type:\"slider\", min:4, max:10, step:1}\r\n","\r\n","\r\n","\r\n","print(\"please wait text is generated...\" ) \r\n","\r\n","gpt2.generate(sess,\r\n","              run_name=give_run_name,\r\n","              length=set_text_length,\r\n","              temperature=set_temperature,\r\n","              prefix=paste_prefix_text,\r\n","              nsamples=set_nr_samples,\r\n","              batch_size=set_nr_samples\r\n","              )"],"execution_count":81,"outputs":[{"output_type":"stream","text":["please wait text is generated...\n","When you don't know what to do.\n","\n","When you’re stuck in traffic.\n","\n","When you’re bored at work.\n","\n","When you’re tired.\n","\n","\n","\n","These are all common complaints I hear from employees. I hear them all the time. “I’m stuck in traffic,” I hear you say. “I can’t get a rhythm here.” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “I can’t get a rhythm here,” I hear it a million times. “Ican’t get a rhythm here,” I hear it a million times. “Ican’t get a rhythm here,” I hear it a million times. “Ican’t get a rhythm here,” I hear it a million times. “Ican’t get a rhythm here,” I hear it a million times. “Ican’t get a rhythm here,” I hear\n","====================\n","When you don't know what to do.\n","\n","When you’re in a bind.\n","\n","When you’re feeling anxious, stressed, or just ready to give up on something.\n","\n","When you’re not.\n","\n","When you’re bored.\n","\n","When your schedule is too full or you’re stretched.\n","\n","\n","\n","\n","\n","How to Handle Tough Feedback\n","\n","When feedback is especially hard on you, there are a few ways you can handle it:\n","\n","Be positive. When you get feedback that you need to respond to, accept it. Think about why you gave the feedback in the first place and how it can help you move forward.\n","\n","Move on. When you’ve accepted the criticism, move on. Give the feedback a name and some context so you can understand its effects.\n","\n","Listen. When someone gives you feedback, acknowledge their input and listen to what you hear.\n","\n","Don’t attack the person. Instead, listen to what they tell you.\n","\n","Wait and see. When you’ve seen enough, move on.\n","\n","\n","\n","\n","\n","How to Be Resilient in the Face of Harsh Criticism\n","\n","When criticism is harsh, you need to be able to take it and turn it into good. Here are some strategies for dealing with it:\n","\n","Be optimistic. When you hear criticism, think about how you want to look forward. Be positive, and look forward to future successes.\n","\n","Take criticism head-on. When someone criticizes, ask yourself, “What do I need to do to be successful in the future?” Look at their actions and actions, not their words.\n","\n","Be flexible. When someone gives you feedback, acknowledge the positive aspects of what you’ve done. Be clear about what you want to change, so you can start to turn things around.\n","\n","Wait and see. When you’ve seen enough, move on.\n","\n","\n","\n","\n","\n","How to Be Confident in Your Thinking\n","\n","You need to be confident in\n","====================\n","When you don't know what to do.”\n","\n","Jamie was silent, and I realized he was trying to avoid the implications of his subordinate’s comments. “You want to do something, you tell me,” I said.\n","\n","“What?”\n","\n","“You want to run something, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to make decisions, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","“You want to lead, you tell me.”\n","\n","Again, silence. “You want to lead, you tell me.”\n","\n","I could see the other team members struggling to understand this new dynamic. I could see them feel as if they were caught between two strong emotions: anger and sadness. I could see them tense up and almost fall apart. I could see their frustration turn to frustration as they tried to figure out how\n","====================\n","When you don't know what to do.\n","\n","When you’re stressed.\n","\n","When you’re bored.\n","\n","\n","\n","So think before you act.\n","\n","When you’re feeling anxious, take a deep breath.\n","\n","When you’re feeling upset, calmly ask for what you need.\n","\n","When you’re feeling anxious because of a situation, step back.\n","\n","When you’re feeling anxious because of a project, take a step back from the situation.\n","\n","When you’re feeling anxious because of a situation, take a step back from the situation.\n","\n","When you’re feeling anxious because of a boss, listen.\n","\n","When you’re feeling anxious because of a group, listen.\n","\n","When you’re feeling anxious because of an organization, step back.\n","\n","When you’re feeling anxious because of someone, listen.\n","\n","\n","\n","When you’re feeling anxious because of someone else, ask yourself, “How might this situation play out for my boss?” You might start the conversation by asking, “How might my boss play this well?” Then, ask yourself, “How might my boss’s boss play this well?” You’ll be playing the role of the spectator while you’re still in the moment. The goal is to be more aware of how others are reacting to you, not to give into insularity.\n","\n","\n","\n","\n","\n","Increase Your Awareness of Others\n","\n","\n","When you’re working with a team, you may be in a bubble and completely focused on your team or a small part of it. This is fine and encouraged, but it doesn’t help you get the most out of the experience. You have to recognize and acknowledge the other people on the team—the facilitators, the coordinators, the line people, and so on. Notice how they act and what they say. What do they look like? What language do they use? What are they wearing? How do they\n","====================\n","When you don't know what to do.\n","\n","If you’re on a team that contains people who think you should get a promotion.\n","\n","If you’re sitting in a conference room with no one else to talk to.\n","\n","If you’re working on a project that seems to be unproductive.\n","\n","If you’re sitting in silence while someone else takes notes.\n","\n","If you’re working on something that nobody has heard yet.\n","\n","If you’re working on something that nobody has thought of.\n","\n","If you’re working on something that nobody has even heard about.\n","\n","If you’re working on something that nobody has any right to care about.\n","\n","\n","\n","\n","\n","How to Speak Up\n","\n","\n","\n","If you don’t speak up when you think you’re being listened to, it can feel as if you’re being listened to less. That’s not the case. If you consistently and loudly state your concerns, your colleagues will likely ignore them. If you raise objections, they might listen but they will most likely ignore you. This dynamic is extremely frustrating to them and makes them hesitant to move forward. So how do you deal with a vocal minority? Be prepared to speak up when you think you’re being heard.\n","\n","First, realize that there are people who don’t like your point of view or your leadership style. That might be hard to figure out, so ask your team for some ideas. Try out a list of people you respect who might be offended by your points of view or your leadership style. Seek out people who will agree with your ideas and then think about what they might be willing to change in order for the team. Be sure to listen first, then ask questions when you’ve started to build a relationship.\n","\n","If you’ve done a good job of listening and posing questions, you should be able to raise your points of view without hesitation or apology. This is not to say you shouldn’t be shy about\n","====================\n"],"name":"stdout"}]}]}