Dataset Editing Tutorial¶
This tutorial demonstrates how to use TokenSmith's powerful Edit Handler to modify and inject content into tokenized datasets. You'll learn how to safely edit training data, inject specific content, and validate modifications before applying them permanently.
Prerequisites:
- Complete tutorials 1-3 (basic setup, inspection, and sampling)
- Have a tokenized dataset ready with batch info generated
- Understanding of tokenization and sequence structure
What you'll learn:
- How to inject text at specific locations in the dataset
- Understanding injection types and their effects
- Safe editing practices with dry runs
- Batch injection workflows for multiple modifications
- Validation and preview techniques before permanent changes
- Best practices for dataset editing and version control
Setup¶
Let's start by setting up our environment and dataset manager, building on the previous tutorials.
In [20]:
Copied!
# Fix paths for imports
import sys
sys.path.insert(0, "/NS/llm-pretraining/work/afkhan/tokensmith")
sys.path.insert(0, "/NS/llm-pretraining/work/afkhan/USC_Colab/gpt-neox")
# Import required libraries
import numpy as np
import random
import warnings
from transformers import AutoTokenizer
from tokensmith.manager import DatasetManager
# Load tokenizer
TOKENIZER_NAME_OR_PATH = "EleutherAI/gpt-neox-20b"
tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_NAME_OR_PATH, add_eos_token=True)
print(f"Loaded tokenizer: {TOKENIZER_NAME_OR_PATH}")
print(f"EOS token: {tokenizer.eos_token} (ID: {tokenizer.eos_token_id})")
# Fix paths for imports
import sys
sys.path.insert(0, "/NS/llm-pretraining/work/afkhan/tokensmith")
sys.path.insert(0, "/NS/llm-pretraining/work/afkhan/USC_Colab/gpt-neox")
# Import required libraries
import numpy as np
import random
import warnings
from transformers import AutoTokenizer
from tokensmith.manager import DatasetManager
# Load tokenizer
TOKENIZER_NAME_OR_PATH = "EleutherAI/gpt-neox-20b"
tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_NAME_OR_PATH, add_eos_token=True)
print(f"Loaded tokenizer: {TOKENIZER_NAME_OR_PATH}")
print(f"EOS token: {tokenizer.eos_token} (ID: {tokenizer.eos_token_id})")
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loaded tokenizer: EleutherAI/gpt-neox-20b EOS token: <|endoftext|> (ID: 0)
In [21]:
Copied!
# Initialize DatasetManager
dataset_manager = DatasetManager()
# Setup the dataset for editing, inspection, sampling, and export
dataset_manager.setup_edit_inspect_sample_export(
dataset_prefix='../../artifacts/data_tokenized_text_document',
batch_info_save_prefix='../../artifacts/batch_info',
train_iters=100,
train_batch_size=16,
train_seq_len=2048,
seed=42,
splits_string='990,5,5',
packing_impl='packed',
allow_chopped=True,
)
print("Dataset manager setup complete!")
print(f"Edit handler available: {dataset_manager.edit is not None}")
# Initialize DatasetManager
dataset_manager = DatasetManager()
# Setup the dataset for editing, inspection, sampling, and export
dataset_manager.setup_edit_inspect_sample_export(
dataset_prefix='../../artifacts/data_tokenized_text_document',
batch_info_save_prefix='../../artifacts/batch_info',
train_iters=100,
train_batch_size=16,
train_seq_len=2048,
seed=42,
splits_string='990,5,5',
packing_impl='packed',
allow_chopped=True,
)
print("Dataset manager setup complete!")
print(f"Edit handler available: {dataset_manager.edit is not None}")
warming up index mmap file...
reading sizes...
reading pointers...
reading document index...
Dataset manager setup complete!
Edit handler available: True
reading sizes...
reading pointers...
reading document index...
Dataset manager setup complete!
Edit handler available: True
Understanding the Edit Handler¶
The Edit Handler provides several key methods for dataset modification:
inject_and_preview()- Inject single text samples with previewinject_multiple_samples()- Batch injection of multiple samplespreview_sample()- Preview existing samples without modificationvalidate_injection_location()- Validate injection locations
Let's explore each of these methods in detail.
Basic Sample Preview¶
Before making any modifications, let's examine existing samples to understand the dataset structure.
In [22]:
Copied!
# Preview a sample without modification
sample_id = 50
# Get sample with document details
sample_text, doc_details = dataset_manager.edit.preview_sample(
sample_id=sample_id,
return_doc_details=True,
return_detokenized=True,
tokenizer=tokenizer
)
print(f"=== Preview of Sample {sample_id} ===")
print(f"Text length: {len(sample_text)} characters")
print(f"Document range: {doc_details['doc_index_f']} to {doc_details['doc_index_l']}")
print(f"Spans multiple docs: {doc_details['doc_index_f'] != doc_details['doc_index_l']}")
print(f"\nSample text (first 200 chars):")
print(f"{sample_text[:200]}...")
print(f"\nSample text (last 100 chars):")
print(f"...{sample_text[-100:]}")
# Preview a sample without modification
sample_id = 50
# Get sample with document details
sample_text, doc_details = dataset_manager.edit.preview_sample(
sample_id=sample_id,
return_doc_details=True,
return_detokenized=True,
tokenizer=tokenizer
)
print(f"=== Preview of Sample {sample_id} ===")
print(f"Text length: {len(sample_text)} characters")
print(f"Document range: {doc_details['doc_index_f']} to {doc_details['doc_index_l']}")
print(f"Spans multiple docs: {doc_details['doc_index_f'] != doc_details['doc_index_l']}")
print(f"\nSample text (first 200 chars):")
print(f"{sample_text[:200]}...")
print(f"\nSample text (last 100 chars):")
print(f"...{sample_text[-100:]}")
=== Preview of Sample 50 === Text length: 8495 characters Document range: 14417 to 14425 Spans multiple docs: True Sample text (first 200 chars): Ben's car fell on the ground and broke. The wheel came off and the paint scratched. "Uh oh!" Lily said, looking at Ben's car. "I'm sorry, Ben. I did not mean to break your car." Ben picked up his car... Sample text (last 100 chars): ...ngs that hid in the dark. She hugged her teddy bear and closed her eyes. She tried to think of happy
In [23]:
Copied!
# Compare with tokenized version
sample_tokens = dataset_manager.edit.preview_sample(
sample_id=sample_id,
return_detokenized=False,
return_doc_details=False
)
total_tokens = sum(len(segment) for segment in sample_tokens)
print(f"Tokenized version:")
print(f"Number of segments: {len(sample_tokens)}")
print(f"Total tokens: {total_tokens}")
print(f"First segment shape: {sample_tokens[0].shape}")
print(f"First 10 tokens: {sample_tokens[0][:10]}")
print(f"Last 10 tokens: {sample_tokens[-1][-10:]}")
# Compare with tokenized version
sample_tokens = dataset_manager.edit.preview_sample(
sample_id=sample_id,
return_detokenized=False,
return_doc_details=False
)
total_tokens = sum(len(segment) for segment in sample_tokens)
print(f"Tokenized version:")
print(f"Number of segments: {len(sample_tokens)}")
print(f"Total tokens: {total_tokens}")
print(f"First segment shape: {sample_tokens[0].shape}")
print(f"First 10 tokens: {sample_tokens[0][:10]}")
print(f"Last 10 tokens: {sample_tokens[-1][-10:]}")
Tokenized version: Number of segments: 9 Total tokens: 2049 First segment shape: (261,) First 10 tokens: [6029 434 1113 6497 327 253 3216 285 9377 15] Last 10 tokens: [4581 617 2927 15 1500 3597 281 1158 273 5211]
Validation and Safety Checks¶
Before performing injections, it's important to validate injection locations and understand dataset boundaries.
In [24]:
Copied!
# Test validation function
test_locations = [0, 50, 100, 500, 1000, 2000, 10000, -1, -5]
print("=== Injection Location Validation ===")
for loc in test_locations:
is_valid = dataset_manager.edit.validate_injection_location(loc)
status = "✓ Valid" if is_valid else "✗ Invalid"
print(f"Location {loc:5d}: {status}")
# Find dataset size
dataset_size = dataset_manager.WriteableMMapIndexedDataset.num_samples
print(f"\nDataset contains {dataset_size} samples")
print(f"Valid injection range: 0 to {dataset_size - 1}")
# Test validation function
test_locations = [0, 50, 100, 500, 1000, 2000, 10000, -1, -5]
print("=== Injection Location Validation ===")
for loc in test_locations:
is_valid = dataset_manager.edit.validate_injection_location(loc)
status = "✓ Valid" if is_valid else "✗ Invalid"
print(f"Location {loc:5d}: {status}")
# Find dataset size
dataset_size = dataset_manager.WriteableMMapIndexedDataset.num_samples
print(f"\nDataset contains {dataset_size} samples")
print(f"Valid injection range: 0 to {dataset_size - 1}")
=== Injection Location Validation === Location 0: ✗ Invalid Location 50: ✗ Invalid Location 100: ✗ Invalid Location 500: ✗ Invalid Location 1000: ✗ Invalid Location 2000: ✗ Invalid Location 10000: ✗ Invalid Location -1: ✗ Invalid Location -5: ✗ Invalid Dataset contains 1600 samples Valid injection range: 0 to 1599
Basic Text Injection with Dry Run¶
Let's start with a basic text injection using dry run mode to safely preview changes.
In [25]:
Copied!
# Basic injection example with dry run
injection_text = "This is a test injection to demonstrate TokenSmith's editing capabilities."
injection_location = 75
print("=== Basic Injection Example (Dry Run) ===")
print(f"Injecting text: '{injection_text}'")
print(f"Location: {injection_location}")
print(f"Injection type: seq_shuffle (default)")
print("\n" + "="*60)
# Perform dry run injection
dataset_manager.edit.inject_and_preview(
text=injection_text,
tokenizer=tokenizer,
injection_loc=injection_location,
injection_type="seq_shuffle",
dry_run=True, # Safe mode - no actual changes
add_eos_token=True
)
# Basic injection example with dry run
injection_text = "This is a test injection to demonstrate TokenSmith's editing capabilities."
injection_location = 75
print("=== Basic Injection Example (Dry Run) ===")
print(f"Injecting text: '{injection_text}'")
print(f"Location: {injection_location}")
print(f"Injection type: seq_shuffle (default)")
print("\n" + "="*60)
# Perform dry run injection
dataset_manager.edit.inject_and_preview(
text=injection_text,
tokenizer=tokenizer,
injection_loc=injection_location,
injection_type="seq_shuffle",
dry_run=True, # Safe mode - no actual changes
add_eos_token=True
)
>> Casting injection data from int64 to <class 'numpy.uint16'>
=== Basic Injection Example (Dry Run) ===
Injecting text: 'This is a test injection to demonstrate TokenSmith's editing capabilities.'
Location: 75
Injection type: seq_shuffle (default)
============================================================
Dummy sample: [ 1552 310 247 1071 8829 281 7568 35097 21484 434 14835 13789
15 0]
Training sample 75
Sample consists of segments from 10 documents
Raw sample: [ 253 16064 15 ... 253 3644 15]
---
Decoded sample: the museum. Sam was very happy that she was able to help BRO when he was broken. The end.<|endoftext|>Once upon a time, there was a little girl named Lily. Lily wanted to know more about magnets. She asked her mom, “What are magnets?” Her mom told her, “Magnets are very impressive. They pull things to them.” Lily was excited. She wanted to see a magnet for herself. She went to the store and found the most incredible magnet. It was a rainbow and it glittered in the light. Lily was so excited and she wanted to show her friends what she knew about magnets. She took out the magnet and showed them how it would attract things. It was so impressive! Her friends were so amazed. They all learned more about magnets. Lily was proud. She now knew a lot about magnets and had an impressive rainbow magnet to show her friends. She was so happy that she had learned something new.<|endoftext|>Once upon a time there was a clumsy bird. His name was Joe. He dreamed of flying with the other birds in the sky. But he was too clumsy for that. One day, Joe saw a beautiful frame hanging from a tree. He flew to it and it was exactly what he needed! He thought that he could use it to make a new flying machine. Joe made a plan and started gathering twigs and pieces of string to make the frame. He worked all day and night to make the frame. The next day, Joe tried to fly with the frame. But it was too heavy. He kept trying but he kept falling down. He felt so disappointed and frustrated. Joe met a wise old bird who had seen his failed attempts to fly. The old bird told him that being clumsy doesn't matter and that he could still fly. Joe realised that he had been focusing too much on the frame and not enough on himself. He started flapping his wings harder and finally he began to fly! The moral of this story is that with enough hard work and dedication, you can achieve your goals too, no matter how clumsy you are.<|endoftext|>Mum had a new table. It was soft and safe. Mum looked at it and smiled. Mum said it was time to pay for the table. She gave the man some money. He said thank you and gave her a big hug. Mum and the little girl put their things on the table. They were so happy. They played with their toys and laughed together. Mum said the table made them feel safe. Mum said it was their special table. The little girl liked to play with her toys at the safe table. Every day when she came home she went straight to it. She was happy to have such a nice safe table.<|endoftext|>Tim and Lily were going to the shore with their mom and dad. They were very happy and enthusiastic. They wanted to play in the sand and water and see the fish and birds. They had to pack their bags with clothes and toys and snacks. Tim was a good boy. He packed his bag quickly and neatly. He put in his swimsuit, his hat, his towel, his ball, his bucket and his shovel. He also packed some crackers and juice for the trip. He helped his mom and dad with the other bags and put them in the car. Lily was a naughty girl. She did not pack her bag at all. She only wanted to play with her dolls and watch TV. She ignored her mom and dad when they told her to pack. She said she did not need anything for the shore. She was lazy and stubborn. When they arrived at the shore, Tim was very happy. He put on his swimsuit and hat and ran to the sand and water. He played with his ball, his bucket and his shovel. He made a big sandcastle and dug a moat. He saw some fish and birds and waved at them. He shared his crackers and juice with his mom and dad. Lily was very sad. She did not have anything to wear or play with. She only had her dress and shoes. She could not go to the sand and water because they were dirty and wet. She could not make a sandcastle or dig a moat because she did not have a bucket or a shovel. She could not see any fish or birds because they were too far away. She was hungry and thirsty but she did not have any crackers or juice. Tim saw Lily sitting alone and crying. He felt sorry for her. He went to her and said, "Lily, why are you sad? Do you want to play with me? You can use my bucket and shovel and ball. You can also have some of my crackers and juice. The shore is fun and beautiful. Come and join me." Lily looked at Tim and smiled. She said, "Thank you, Tim. You are very kind and generous. I am sorry I was naughty and did not pack. I was wrong and foolish. I missed out on a lot of fun and joy. Can you forgive me?" Tim hugged Lily and said, "Of course I forgive you, Lily. You are my sister and I love you. Come on, let's go and have fun together. The shore is big enough for both of us." Lily and Tim went to the sand and water and played together. They made a bigger sandcastle and dug a deeper moat. They saw more fish and birds and talked to them. They shared their crackers and juice and laughed. They had a wonderful time at the shore. Mom and dad watched them and smiled. They were proud of Tim and happy for Lily. They learned a lesson that day. They learned that it is better to pack and be prepared than to be lazy and unprepared. They learned that it is better to share and be kind than to be selfish and stubborn. They learned that the shore is a place of fun and beauty for those who are enthusiastic and grateful.<|endoftext|>Once there was a boy named Jay. He was 3 years old. He wanted to go to a very impressive place. So he started walking. After walking for a while, Jay had gone a long way but he still hadn't seen anything special. Then he saw a building that was so tall and it had zero windows. He thought it was very impressive. He wanted to go inside, so he went up to the door. He knocked and knocked but no one answered. Jay thought maybe it was closed and he didn't know what to do. Just then he saw a ladder leaning against the building. It was he only way to go inside. Slowly and carefully, Jay started climbing up the ladder. He climbed higher and higher until he finally reached the top. When he opened the door he couldn't believe what he saw. It was an amazing room with lots of cool things inside. Jay was so happy! He stayed in the room for hours. Eventually it was time to go home, so he carefully climbed down the ladder. He was very happy that he had gone to such an impressive place.<|endoftext|>Once upon a time there was an obedient boy. He wanted to go outside and play. So he got dressed and ran out the door. But his mom said he had to stay still. The boy was confused. He wanted to go. But his mom said no. Then he heard a sound outside. It was a loud loud noise! The boy was scared. His mom then took him to the window and showed him the gun. The gun was very big and the boy was worried. But his mom said it was okay. The gun was just there to protect them from danger. The boy felt better. He knew that he had to be obedient if he wanted to go outside. So the boy stayed close to his mom and was very good. Finally his mom said he could go out and play. The boy smiled and ran outside. He was so happy to be able to go play. He ran and ran until he couldn't run anymore. He had a lot of fun and was obedient the whole time.<|endoftext|>Once upon a time, there was a brave giraffe called Jojo who was eager to go on a search. Jojo was feeling very proud, so she told her friends. “I’m going on a search. Who wants to join me?” The other animals were scared, so they decided to stay put. But Jojo was determined, so she set off alone. As she ventured deeper and deeper into the forest, she came across a big rock. Suddenly, a voice came from behind it. It was a squirrel, and he had an important question for Jojo. “Do you want to discuss something?” he said. Jojo thought for a moment, then replied, “Yes, I would like that.” So they began to discuss the best way to go on a search together. Eventually, the pair decided to work as a team. They set off, exploring and searching all around the forest. Jojo felt so proud when they eventually found what they were looking for. The pair became great friends, and Jojo knew she would always be proud of their search.<|endoftext|>One day, a little girl named Lucy found some shiny jewelry in her mom's room. She was very excited. Lucy put the jewelry in her pocket and went outside to play. While playing, Lucy saw a big, scary dog. The dog saw the jewelry in her pocket and wanted it. The dog said, "Give me the jewelry, or I will chase you!" Lucy was scared and didn't want to give the jewelry to the dog. Lucy tried to escape from the dog, but she tripped and fell. The jewelry fell out of her pocket. The big, scary dog took the jewelry and ran away. Lucy was very sad and cried. She knew she should not have taken her mom's jewelry.<|endoftext|>Once upon a time there was a girl. She was nosy and liked to explore things. She had a flashlight that she liked to grab and find things with in the dark.
---
Training sample 75 after injection
Raw sample: [ 253 16064 15 ... 253 3644 15]
---
Decoded sample: the museum. Sam was very happy that she was able to help BRO when he was broken. The end.<|endoftext|>Once upon a time, there was a little girl named Lily. Lily wanted to know more about magnets. She asked her mom, “What are magnets?” Her mom told her, “Magnets are very impressive. They pull things to them.” Lily was excited. She wanted to see a magnet for herself. She went to the store and found the most incredible magnet. It was a rainbow and it glittered in the light. Lily was so excited and she wanted to show her friends what she knew about magnets. She took out the magnet and showed them how it would attract things. It was so impressive! Her friends were so amazed. They all learned more about magnets. Lily was proud. She now knew a lot about magnets and had an impressive rainbow magnet to show her friends. She was so happy that she had learned something new.<|endoftext|>Once upon a time there was a clumsy bird. His name was Joe. He dreamed of flying with the other birds in the sky. But he was too clumsy for that. One day, Joe saw a beautiful frame hanging from a tree. He flew to it and it was exactly what he needed! He thought that he could use it to make a new flying machine. Joe made a plan and started gathering twigs and pieces of string to make the frame. He worked all day and night to make the frame. The next day, Joe tried to fly with the frame. But it was too heavy. He kept trying but he kept falling down. He felt so disappointed and frustrated. Joe met a wise old bird who had seen his failed attempts to fly. The old bird told him that being clumsy doesn't matter and that he could still fly. Joe realised that he had been focusing too much on the frame and not enough on himself. He started flapping his wings harder and finally he began to fly! The moral of this story is that with enough hard work and dedication, you can achieve your goals too, no matter how clumsy you are.<|endoftext|>Mum had a new table. It was soft and safe. Mum looked at it and smiled. Mum said it was time to pay for the table. She gave the man some money. He said thank you and gave her a big hug. Mum and the little girl put their things on the table. They were so happy. They played with their toys and laughed together. Mum said the table made them feel safe. Mum said it was their special table. The little girl liked to play with her toys at the safe table. Every day when she came home she went straight to it. She was happy to have such a nice safe table.<|endoftext|>Tim and Lily were going to the shore with their mom and dad. They were very happy and enthusiastic. They wanted to play in the sand and water and see the fish and birds. They had to pack their bags with clothes and toys and snacks. Tim was a good boy. He packed his bag quickly and neatly. He put in his swimsuit, his hat, his towel, his ball, his bucket and his shovel. He also packed some crackers and juice for the trip. He helped his mom and dad with the other bags and put them in the car. Lily was a naughty girl. She did not pack her bag at all. She only wanted to play with her dolls and watch TV. She ignored her mom and dad when they told her to pack. She said she did not need anything for the shore. She was lazy and stubborn. When they arrived at the shore, Tim was very happy. He put on his swimsuit and hat and ran to the sand and water. He played with his ball, his bucket and his shovel. He made a big sandcastle and dug a moat. He saw some fish and birds and waved at them. He shared his crackers and juice with his mom and dad. Lily was very sad. She did not have anything to wear or play with. She only had her dress and shoes. She could not go to the sand and water because they were dirty and wet. She could not make a sandcastle or dig a moat because she did not have a bucket or a shovel. She could not see any fish or birds because they were too far away. She was hungry and thirsty but she did not have any crackers or juice. Tim saw Lily sitting alone and crying. He felt sorry for her. He went to her and said, "Lily, why are you sad? Do you want to play with me? You can use my bucket and shovel and ball. You can also have some of my crackers and juice. The shore is fun and beautiful. Come and join me." Lily looked at Tim and smiled. She said, "Thank you, Tim. You are very kind and generous. I am sorry I was naughty and did not pack. I was wrong and foolish. I missed out on a lot of fun and joy. Can you forgive me?" Tim hugged Lily and said, "Of course I forgive you, Lily. You are my sister and I love you. Come on, let's go and have fun together. The shore is big enough for both of us." Lily and Tim went to the sand and water and played together. They made a bigger sandcastle and dug a deeper moat. They saw more fish and birds and talked to them. They shared their crackers and juice and laughed. They had a wonderful time at the shore. Mom and dad watched them and smiled. They were proud of Tim and happy for Lily. They learned a lesson that day. They learned that it is better to pack and be prepared than to be lazy and unprepared. They learned that it is better to share and be kind than to be selfish and stubborn. They learned that the shore is a place of fun and beauty for those who are enthusiastic and grateful.<|endoftext|>Once there was a boy named Jay. He was 3 years old. He wanted to go to a very impressive place. So he started walking. After walking for a while, Jay had gone a long way but he still hadn't seen anything special. Then he saw a building that was so tall and it had zero windows. He thought it was very impressive. He wanted to go inside, so he went up to the door. He knocked and knocked but no one answered. Jay thought maybe it was closed and he didn't know what to do. Just then he saw a ladder leaning against the building. It was he only way to go inside. Slowly and carefully, Jay started climbing up the ladder. He climbed higher and higher until he finally reached the top. When he opened the door he couldn't believe what he saw. It was an amazing room with lots of cool things inside. Jay was so happy! He stayed in the room for hours. Eventually it was time to go home, so he carefully climbed down the ladder. He was very happy that he had gone to such an impressive place.<|endoftext|>Once upon a time there was an obedient boy. He wanted to go outside and play. So he got dressed and ran out the door. But his mom said he had to stay still. The boy was confused. He wanted to go. But his mom said no. Then he heard a sound outside. It was a loud loud noise! The boy was scared. His mom then took him to the window and showed him the gun. The gun was very big and the boy was worried. But his mom said it was okay. The gun was just there to protect them from danger. The boy felt better. He knew that he had to be obedient if he wanted to go outside. So the boy stayed close to his mom and was very good. Finally his mom said he could go out and play. The boy smiled and ran outside. He was so happy to be able to go play. He ran and ran until he couldn't run anymore. He had a lot of fun and was obedient the whole time.<|endoftext|>Once upon a time, there was a brave giraffe called Jojo who was eager to go on a search. Jojo was feeling very proud, so she told her friends. “I’m going on a search. Who wants to join me?” The other animals were scared, so they decided to stay put. But Jojo was determined, so she set off alone. As she ventured deeper and deeper into the forest, she came across a big rock. Suddenly, a voice came from behind it. It was a squirrel, and he had an important question for Jojo. “Do you want to discuss something?” he said. Jojo thought for a moment, then replied, “Yes, I would like that.” So they began to discuss the best way to go on a search together. Eventually, the pair decided to work as a team. They set off, exploring and searching all around the forest. Jojo felt so proud when they eventually found what they were looking for. The pair became great friends, and Jojo knew she would always be proud of their search.<|endoftext|>One day, a little girl named Lucy found some shiny jewelry in her mom's room. She was very excited. Lucy put the jewelry in her pocket and went outside to play. While playing, Lucy saw a big, scary dog. The dog saw the jewelry in her pocket and wanted it. The dog said, "Give me the jewelry, or I will chase you!" Lucy was scared and didn't want to give the jewelry to the dog. Lucy tried to escape from the dog, but she tripped and fell. The jewelry fell out of her pocket. The big, scary dog took the jewelry and ran away. Lucy was very sad and cried. She knew she should not have taken her mom's jewelry.<|endoftext|>Once upon a time there was a girl. She was nosy and liked to explore things. She had a flashlight that she liked to grab and find things with in the dark.
---
Understanding Injection Types¶
TokenSmith supports two injection types that determine where in the sequence the new content is placed:
seq_shuffle- Randomly places the injection within the sequenceseq_start- Places the injection at the beginning of the sequence
Let's compare both types:
In [26]:
Copied!
# Compare injection types
test_text = "INJECTION: This text demonstrates different injection strategies."
test_location = 125
print("=== Comparing Injection Types ===")
print(f"Test text: '{test_text}'")
print(f"Location: {test_location}")
# Test seq_shuffle injection
print("\n" + "="*50)
print("1. SEQ_SHUFFLE Injection:")
print("="*50)
result_shuffle = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
dry_run=True,
return_details=True,
rng=np.random.default_rng(42) # Fixed seed for reproducibility
)
print(f"Original length: {len(result_shuffle['original_sample']['decoded_text'])}")
print(f"Modified length: {len(result_shuffle['modified_sample']['decoded_text'])}")
print(f"Injection position determined by: Random placement within sequence")
# Compare injection types
test_text = "INJECTION: This text demonstrates different injection strategies."
test_location = 125
print("=== Comparing Injection Types ===")
print(f"Test text: '{test_text}'")
print(f"Location: {test_location}")
# Test seq_shuffle injection
print("\n" + "="*50)
print("1. SEQ_SHUFFLE Injection:")
print("="*50)
result_shuffle = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
dry_run=True,
return_details=True,
rng=np.random.default_rng(42) # Fixed seed for reproducibility
)
print(f"Original length: {len(result_shuffle['original_sample']['decoded_text'])}")
print(f"Modified length: {len(result_shuffle['modified_sample']['decoded_text'])}")
print(f"Injection position determined by: Random placement within sequence")
>> Casting injection data from int64 to <class 'numpy.uint16'>
=== Comparing Injection Types === Test text: 'INJECTION: This text demonstrates different injection strategies.' Location: 125 ================================================== 1. SEQ_SHUFFLE Injection: ================================================== Original length: 8419 Modified length: 8419 Injection position determined by: Random placement within sequence
In [27]:
Copied!
# Test seq_start injection
print("\n" + "="*50)
print("2. SEQ_START Injection:")
print("="*50)
result_start = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_start",
dry_run=True,
return_details=True
)
print(f"Original length: {len(result_start['original_sample']['decoded_text'])}")
print(f"Modified length: {len(result_start['modified_sample']['decoded_text'])}")
print(f"Injection position: Beginning of sequence")
# Show first 200 characters to see the injection
print(f"\nFirst 200 chars of modified sample:")
print(f"{result_start['modified_sample']['decoded_text'][:200]}...")
# Test seq_start injection
print("\n" + "="*50)
print("2. SEQ_START Injection:")
print("="*50)
result_start = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_start",
dry_run=True,
return_details=True
)
print(f"Original length: {len(result_start['original_sample']['decoded_text'])}")
print(f"Modified length: {len(result_start['modified_sample']['decoded_text'])}")
print(f"Injection position: Beginning of sequence")
# Show first 200 characters to see the injection
print(f"\nFirst 200 chars of modified sample:")
print(f"{result_start['modified_sample']['decoded_text'][:200]}...")
>> Casting injection data from int64 to <class 'numpy.uint16'>
================================================== 2. SEQ_START Injection: ================================================== Original length: 8419 Modified length: 8419 Injection position: Beginning of sequence First 200 chars of modified sample: a time there was a little ice cream cone. It was filled with white, creamy ice cream, and it made the cone look normal. But then something strange happened. The ice cream started to melt. It oozed an...
Advanced Injection with Return Details¶
For programmatic analysis, we can return structured data instead of just printing results.
In [28]:
Copied!
# Advanced injection with detailed analysis
analysis_text = "ANALYSIS: This injection includes detailed metadata for research purposes."
analysis_location = 200
print("=== Advanced Injection Analysis ===")
injection_result = dataset_manager.edit.inject_and_preview(
text=analysis_text,
tokenizer=tokenizer,
injection_loc=analysis_location,
injection_type="seq_shuffle",
dry_run=True,
return_details=True,
add_eos_token=True,
rng=np.random.default_rng(123)
)
# Analyze the results
print(f"Injection Location: {injection_result['injection_location']}")
print(f"Injection Type: {injection_result['injection_type']}")
print(f"Dry Run: {injection_result['dry_run']}")
print(f"Injected Text: '{injection_result['injected_text']}'")
print(f"Injected Tokens: {len(injection_result['injected_tokens'])} tokens")
print(f"First 10 injected tokens: {injection_result['injected_tokens'][:10]}")
# Compare original vs modified
orig = injection_result['original_sample']
mod = injection_result['modified_sample']
print(f"\nOriginal Sample:")
print(f" Token count: {len(orig['raw_tokens'])}")
print(f" Character count: {len(orig['decoded_text'])}")
print(f" Document spans: {orig['num_documents']}")
print(f"\nModified Sample:")
print(f" Token count: {len(mod['raw_tokens'])}")
print(f" Character count: {len(mod['decoded_text'])}")
print(f" Document spans: {mod['num_documents']}")
# Calculate changes
token_diff = len(mod['raw_tokens']) - len(orig['raw_tokens'])
char_diff = len(mod['decoded_text']) - len(orig['decoded_text'])
print(f"\nChanges:")
print(f" Token difference: {token_diff:+d}")
print(f" Character difference: {char_diff:+d}")
# Advanced injection with detailed analysis
analysis_text = "ANALYSIS: This injection includes detailed metadata for research purposes."
analysis_location = 200
print("=== Advanced Injection Analysis ===")
injection_result = dataset_manager.edit.inject_and_preview(
text=analysis_text,
tokenizer=tokenizer,
injection_loc=analysis_location,
injection_type="seq_shuffle",
dry_run=True,
return_details=True,
add_eos_token=True,
rng=np.random.default_rng(123)
)
# Analyze the results
print(f"Injection Location: {injection_result['injection_location']}")
print(f"Injection Type: {injection_result['injection_type']}")
print(f"Dry Run: {injection_result['dry_run']}")
print(f"Injected Text: '{injection_result['injected_text']}'")
print(f"Injected Tokens: {len(injection_result['injected_tokens'])} tokens")
print(f"First 10 injected tokens: {injection_result['injected_tokens'][:10]}")
# Compare original vs modified
orig = injection_result['original_sample']
mod = injection_result['modified_sample']
print(f"\nOriginal Sample:")
print(f" Token count: {len(orig['raw_tokens'])}")
print(f" Character count: {len(orig['decoded_text'])}")
print(f" Document spans: {orig['num_documents']}")
print(f"\nModified Sample:")
print(f" Token count: {len(mod['raw_tokens'])}")
print(f" Character count: {len(mod['decoded_text'])}")
print(f" Document spans: {mod['num_documents']}")
# Calculate changes
token_diff = len(mod['raw_tokens']) - len(orig['raw_tokens'])
char_diff = len(mod['decoded_text']) - len(orig['decoded_text'])
print(f"\nChanges:")
print(f" Token difference: {token_diff:+d}")
print(f" Character difference: {char_diff:+d}")
>> Casting injection data from int64 to <class 'numpy.uint16'>
=== Advanced Injection Analysis === Injection Location: 200 Injection Type: seq_shuffle Dry Run: True Injected Text: 'ANALYSIS: This injection includes detailed metadata for research purposes.' Injected Tokens: 15 tokens First 10 injected tokens: [34, 21686, 6328, 1830, 27, 831, 8829, 3797, 7000, 21464] Original Sample: Token count: 2049 Character count: 8590 Document spans: 13 Modified Sample: Token count: 2049 Character count: 8590 Document spans: 13 Changes: Token difference: +0 Character difference: +0
Batch Injection Workflows¶
For research and analysis, you often need to inject multiple samples. The Edit Handler supports batch operations.
In [29]:
Copied!
# Prepare multiple injections
injections = [
{
"text": "PROMPT: Once upon a time in a digital kingdom,",
"injection_loc": 300,
"injection_type": "seq_start"
},
{
"text": "CONTEXT: This story explores the intersection of technology and narrative.",
"injection_loc": 301,
"injection_type": "seq_shuffle"
},
{
"text": "INSTRUCTION: Please continue this story with creative and engaging content.",
"injection_loc": 302,
"injection_type": "seq_start"
},
{
"text": "METADATA: Generated by TokenSmith for research purposes.",
"injection_loc": 303,
"injection_type": "seq_shuffle"
}
]
print("=== Batch Injection Example ===")
print(f"Prepared {len(injections)} injections:")
for i, inj in enumerate(injections, 1):
print(f" {i}. Location {inj['injection_loc']:3d} ({inj['injection_type']}): '{inj['text'][:50]}...'")
print("\n" + "="*70)
# Prepare multiple injections
injections = [
{
"text": "PROMPT: Once upon a time in a digital kingdom,",
"injection_loc": 300,
"injection_type": "seq_start"
},
{
"text": "CONTEXT: This story explores the intersection of technology and narrative.",
"injection_loc": 301,
"injection_type": "seq_shuffle"
},
{
"text": "INSTRUCTION: Please continue this story with creative and engaging content.",
"injection_loc": 302,
"injection_type": "seq_start"
},
{
"text": "METADATA: Generated by TokenSmith for research purposes.",
"injection_loc": 303,
"injection_type": "seq_shuffle"
}
]
print("=== Batch Injection Example ===")
print(f"Prepared {len(injections)} injections:")
for i, inj in enumerate(injections, 1):
print(f" {i}. Location {inj['injection_loc']:3d} ({inj['injection_type']}): '{inj['text'][:50]}...'")
print("\n" + "="*70)
=== Batch Injection Example === Prepared 4 injections: 1. Location 300 (seq_start): 'PROMPT: Once upon a time in a digital kingdom,...' 2. Location 301 (seq_shuffle): 'CONTEXT: This story explores the intersection of t...' 3. Location 302 (seq_start): 'INSTRUCTION: Please continue this story with creat...' 4. Location 303 (seq_shuffle): 'METADATA: Generated by TokenSmith for research pur...' ======================================================================
In [30]:
Copied!
# Execute batch injections with detailed results
batch_results = dataset_manager.edit.inject_multiple_samples(
injections=injections,
tokenizer=tokenizer,
rng=np.random.default_rng(456),
add_eos_token=True,
dry_run=True, # Safe mode
return_details=True
)
print(f"\nBatch injection completed! Processed {len(batch_results)} injections.")
# Analyze batch results
print("\n=== Batch Results Summary ===")
for i, result in enumerate(batch_results, 1):
if 'error' in result:
print(f"Injection {i}: ERROR - {result['error']}")
else:
orig_len = len(result['original_sample']['decoded_text'])
mod_len = len(result['modified_sample']['decoded_text'])
diff = mod_len - orig_len
print(f"Injection {i}: SUCCESS - Location {result['injection_location']}, +{diff} chars")
# Execute batch injections with detailed results
batch_results = dataset_manager.edit.inject_multiple_samples(
injections=injections,
tokenizer=tokenizer,
rng=np.random.default_rng(456),
add_eos_token=True,
dry_run=True, # Safe mode
return_details=True
)
print(f"\nBatch injection completed! Processed {len(batch_results)} injections.")
# Analyze batch results
print("\n=== Batch Results Summary ===")
for i, result in enumerate(batch_results, 1):
if 'error' in result:
print(f"Injection {i}: ERROR - {result['error']}")
else:
orig_len = len(result['original_sample']['decoded_text'])
mod_len = len(result['modified_sample']['decoded_text'])
diff = mod_len - orig_len
print(f"Injection {i}: SUCCESS - Location {result['injection_location']}, +{diff} chars")
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
Batch injection completed! Processed 4 injections. === Batch Results Summary === Injection 1: SUCCESS - Location 300, +0 chars Injection 2: SUCCESS - Location 301, +0 chars Injection 3: SUCCESS - Location 302, +0 chars Injection 4: SUCCESS - Location 303, +0 chars
Reproducible Injection with Seeds¶
For research reproducibility, it's important to control randomness in injections.
In [31]:
Copied!
# Demonstrate reproducible injections
def reproducible_injection_demo():
"""Demonstrate that same seeds produce identical injection results."""
test_text = "REPRODUCIBILITY: This injection should be identical across runs."
test_location = 150
seed = 789
print("=== Reproducibility Test ===")
# First injection
result1 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed),
dry_run=True,
return_details=True
)
# Second injection with same seed
result2 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed), # Same seed
dry_run=True,
return_details=True
)
# Third injection with different seed
result3 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed + 1), # Different seed
dry_run=True,
return_details=True
)
# Compare results
identical_12 = result1['modified_sample']['decoded_text'] == result2['modified_sample']['decoded_text']
identical_13 = result1['modified_sample']['decoded_text'] == result3['modified_sample']['decoded_text']
print(f"Result 1 == Result 2 (same seed): {identical_12}")
print(f"Result 1 == Result 3 (different seed): {identical_13}")
# Show injection details for verification
for i, result in enumerate([result1, result2, result3], 1):
details = result['injection_details']
print(f"\nRun {i} injection details:")
if 'pt_window_offset' in details:
print(f" Window offset: {details['pt_window_offset']}")
if 'pt_injection_len' in details:
print(f" Injection length: {details['pt_injection_len']}")
reproducible_injection_demo()
# Demonstrate reproducible injections
def reproducible_injection_demo():
"""Demonstrate that same seeds produce identical injection results."""
test_text = "REPRODUCIBILITY: This injection should be identical across runs."
test_location = 150
seed = 789
print("=== Reproducibility Test ===")
# First injection
result1 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed),
dry_run=True,
return_details=True
)
# Second injection with same seed
result2 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed), # Same seed
dry_run=True,
return_details=True
)
# Third injection with different seed
result3 = dataset_manager.edit.inject_and_preview(
text=test_text,
tokenizer=tokenizer,
injection_loc=test_location,
injection_type="seq_shuffle",
rng=np.random.default_rng(seed + 1), # Different seed
dry_run=True,
return_details=True
)
# Compare results
identical_12 = result1['modified_sample']['decoded_text'] == result2['modified_sample']['decoded_text']
identical_13 = result1['modified_sample']['decoded_text'] == result3['modified_sample']['decoded_text']
print(f"Result 1 == Result 2 (same seed): {identical_12}")
print(f"Result 1 == Result 3 (different seed): {identical_13}")
# Show injection details for verification
for i, result in enumerate([result1, result2, result3], 1):
details = result['injection_details']
print(f"\nRun {i} injection details:")
if 'pt_window_offset' in details:
print(f" Window offset: {details['pt_window_offset']}")
if 'pt_injection_len' in details:
print(f" Injection length: {details['pt_injection_len']}")
reproducible_injection_demo()
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
=== Reproducibility Test === Result 1 == Result 2 (same seed): True Result 1 == Result 3 (different seed): True Run 1 injection details: Window offset: 1 Injection length: 15 Run 2 injection details: Window offset: 1 Injection length: 15 Run 3 injection details: Window offset: 0 Injection length: 15
Safe Editing Practices¶
Let's explore best practices for safe dataset editing, including validation, backup strategies, and incremental testing.
In [32]:
Copied!
# Comprehensive safety check function
def comprehensive_safety_check(injections_list, tokenizer, dataset_manager):
"""Perform comprehensive safety checks before batch injection."""
print("=== Comprehensive Safety Check ===")
# Check 1: Validate all injection locations
print("\n1. Validating injection locations...")
invalid_locations = []
for i, inj in enumerate(injections_list):
loc = inj['injection_loc']
if not dataset_manager.edit.validate_injection_location(loc):
invalid_locations.append((i, loc))
if invalid_locations:
print(f" ❌ Found {len(invalid_locations)} invalid locations:")
for idx, loc in invalid_locations:
print(f" Injection {idx}: location {loc}")
return False
else:
print(f" ✅ All {len(injections_list)} locations are valid")
# Check 2: Validate injection texts
print("\n2. Validating injection texts...")
empty_texts = []
for i, inj in enumerate(injections_list):
if not inj.get('text') or not inj['text'].strip():
empty_texts.append(i)
if empty_texts:
print(f" ❌ Found {len(empty_texts)} empty texts at indices: {empty_texts}")
return False
else:
print(f" ✅ All texts are non-empty")
# Check 3: Test tokenization
print("\n3. Testing tokenization...")
tokenization_errors = []
for i, inj in enumerate(injections_list):
try:
tokens = tokenizer(inj['text'])['input_ids']
if len(tokens) == 0:
tokenization_errors.append((i, "Empty token sequence"))
except Exception as e:
tokenization_errors.append((i, str(e)))
if tokenization_errors:
print(f" ❌ Found {len(tokenization_errors)} tokenization errors:")
for idx, error in tokenization_errors:
print(f" Injection {idx}: {error}")
return False
else:
print(f" ✅ All texts tokenize successfully")
# Check 4: Preview first injection
print("\n4. Previewing first injection...")
try:
first_inj = injections_list[0]
preview_result = dataset_manager.edit.inject_and_preview(
text=first_inj['text'],
tokenizer=tokenizer,
injection_loc=first_inj['injection_loc'],
injection_type=first_inj.get('injection_type', 'seq_shuffle'),
dry_run=True,
return_details=True
)
print(f" ✅ Preview successful for location {first_inj['injection_loc']}")
except Exception as e:
print(f" ❌ Preview failed: {e}")
return False
print("\n🎉 All safety checks passed!")
return True
# Test with our injection list
safety_result = comprehensive_safety_check(injections, tokenizer, dataset_manager)
print(f"\nSafety check result: {'PASSED' if safety_result else 'FAILED'}")
# Comprehensive safety check function
def comprehensive_safety_check(injections_list, tokenizer, dataset_manager):
"""Perform comprehensive safety checks before batch injection."""
print("=== Comprehensive Safety Check ===")
# Check 1: Validate all injection locations
print("\n1. Validating injection locations...")
invalid_locations = []
for i, inj in enumerate(injections_list):
loc = inj['injection_loc']
if not dataset_manager.edit.validate_injection_location(loc):
invalid_locations.append((i, loc))
if invalid_locations:
print(f" ❌ Found {len(invalid_locations)} invalid locations:")
for idx, loc in invalid_locations:
print(f" Injection {idx}: location {loc}")
return False
else:
print(f" ✅ All {len(injections_list)} locations are valid")
# Check 2: Validate injection texts
print("\n2. Validating injection texts...")
empty_texts = []
for i, inj in enumerate(injections_list):
if not inj.get('text') or not inj['text'].strip():
empty_texts.append(i)
if empty_texts:
print(f" ❌ Found {len(empty_texts)} empty texts at indices: {empty_texts}")
return False
else:
print(f" ✅ All texts are non-empty")
# Check 3: Test tokenization
print("\n3. Testing tokenization...")
tokenization_errors = []
for i, inj in enumerate(injections_list):
try:
tokens = tokenizer(inj['text'])['input_ids']
if len(tokens) == 0:
tokenization_errors.append((i, "Empty token sequence"))
except Exception as e:
tokenization_errors.append((i, str(e)))
if tokenization_errors:
print(f" ❌ Found {len(tokenization_errors)} tokenization errors:")
for idx, error in tokenization_errors:
print(f" Injection {idx}: {error}")
return False
else:
print(f" ✅ All texts tokenize successfully")
# Check 4: Preview first injection
print("\n4. Previewing first injection...")
try:
first_inj = injections_list[0]
preview_result = dataset_manager.edit.inject_and_preview(
text=first_inj['text'],
tokenizer=tokenizer,
injection_loc=first_inj['injection_loc'],
injection_type=first_inj.get('injection_type', 'seq_shuffle'),
dry_run=True,
return_details=True
)
print(f" ✅ Preview successful for location {first_inj['injection_loc']}")
except Exception as e:
print(f" ❌ Preview failed: {e}")
return False
print("\n🎉 All safety checks passed!")
return True
# Test with our injection list
safety_result = comprehensive_safety_check(injections, tokenizer, dataset_manager)
print(f"\nSafety check result: {'PASSED' if safety_result else 'FAILED'}")
=== Comprehensive Safety Check ===
1. Validating injection locations...
❌ Found 4 invalid locations:
Injection 0: location 300
Injection 1: location 301
Injection 2: location 302
Injection 3: location 303
Safety check result: FAILED
Performance Considerations¶
When editing large datasets, performance becomes important. Let's explore efficient editing strategies.
In [33]:
Copied!
import time
# Performance testing for different injection strategies
def performance_test():
"""Test performance of different injection approaches."""
print("=== Performance Testing ===")
# Test 1: Single injections
print("\n1. Testing single injections...")
single_injection_times = []
for i in range(5):
start_time = time.time()
dataset_manager.edit.inject_and_preview(
text=f"Performance test injection {i}",
tokenizer=tokenizer,
injection_loc=400 + i,
injection_type="seq_shuffle",
dry_run=True,
return_details=False # Skip detailed output for speed
)
single_injection_times.append(time.time() - start_time)
avg_single = np.mean(single_injection_times)
print(f" Average single injection time: {avg_single:.4f} seconds")
# Test 2: Batch injection
print("\n2. Testing batch injection...")
batch_injections = [
{
"text": f"Batch performance test {i}",
"injection_loc": 500 + i,
"injection_type": "seq_shuffle"
}
for i in range(5)
]
start_time = time.time()
dataset_manager.edit.inject_multiple_samples(
injections=batch_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=False
)
batch_time = time.time() - start_time
print(f" Batch injection time (5 injections): {batch_time:.4f} seconds")
print(f" Average per injection in batch: {batch_time / 5:.4f} seconds")
# Performance comparison
print("\n3. Performance comparison:")
total_single_time = avg_single * 5
speedup = total_single_time / batch_time if batch_time > 0 else float('inf')
print(f" 5 single injections: {total_single_time:.4f} seconds")
print(f" 1 batch injection: {batch_time:.4f} seconds")
print(f" Speedup factor: {speedup:.2f}x")
performance_test()
import time
# Performance testing for different injection strategies
def performance_test():
"""Test performance of different injection approaches."""
print("=== Performance Testing ===")
# Test 1: Single injections
print("\n1. Testing single injections...")
single_injection_times = []
for i in range(5):
start_time = time.time()
dataset_manager.edit.inject_and_preview(
text=f"Performance test injection {i}",
tokenizer=tokenizer,
injection_loc=400 + i,
injection_type="seq_shuffle",
dry_run=True,
return_details=False # Skip detailed output for speed
)
single_injection_times.append(time.time() - start_time)
avg_single = np.mean(single_injection_times)
print(f" Average single injection time: {avg_single:.4f} seconds")
# Test 2: Batch injection
print("\n2. Testing batch injection...")
batch_injections = [
{
"text": f"Batch performance test {i}",
"injection_loc": 500 + i,
"injection_type": "seq_shuffle"
}
for i in range(5)
]
start_time = time.time()
dataset_manager.edit.inject_multiple_samples(
injections=batch_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=False
)
batch_time = time.time() - start_time
print(f" Batch injection time (5 injections): {batch_time:.4f} seconds")
print(f" Average per injection in batch: {batch_time / 5:.4f} seconds")
# Performance comparison
print("\n3. Performance comparison:")
total_single_time = avg_single * 5
speedup = total_single_time / batch_time if batch_time > 0 else float('inf')
print(f" 5 single injections: {total_single_time:.4f} seconds")
print(f" 1 batch injection: {batch_time:.4f} seconds")
print(f" Speedup factor: {speedup:.2f}x")
performance_test()
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
=== Performance Testing === 1. Testing single injections... Dummy sample: [35975 1071 8829 470 0] Training sample 400 Sample consists of segments from 10 documents Raw sample: [ 1476 16543 12918 ... 452 794 326] --- Decoded sample: !" Anna laughed. She saw a dress and a scarf and put them on. "And I look like a queen!" They pretended to be a king and a queen and had fun. They sat on a seat in the closet and talked about their kingdom. But then they heard a voice. It was Mom. She was looking for them. "Anna! Ben! Where are you?" Mom called. Anna and Ben got scared. They did not want Mom to see them in the old clothes. They thought Mom would be angry. "Quick, hide!" Anna whispered. They took off the clothes and hats and put them back in the closet. They closed the door and stayed quiet. Mom came to the hall. She saw the closet and opened it. She was surprised to see Anna and Ben inside. "What are you doing here?" Mom asked. Anna and Ben looked at Mom. They did not know what to say. "We were... we were..." Ben tried to explain. "We were playing king and queen!" Anna said. Mom smiled. She was not angry. She was amused. "King and queen, huh? That's a nice game. But you know, these clothes are very ancient. They belong to Grandma and Grandpa. They are very special and fragile. You have to be careful with them." Anna and Ben nodded. They felt sorry. "We're sorry, Mom. We didn't know. We just wanted to have fun," Anna said. Mom hugged them. She was not mad. "It's okay, sweeties. I know you didn't mean any harm. But next time, ask me before you play with the old things. Okay?" "Okay, Mom," Anna and Ben said. Mom took them out of the closet and gave them a kiss. "Now, how about we go and have some cookies and milk? And maybe later, I can tell you some stories about Grandma and Grandpa and their ancient clothes. Would you like that?" Anna and Ben smiled. They liked stories. "Yes, please, Mom!" they said. They followed Mom to the kitchen. They were happy. They had a good day.<|endoftext|>Teddy was a little bear who loved to play the piano. He loved to hear it's orange keys and make funny sounds. One day, Teddy wanted to have a special day of playing. He wanted to show off to all his friends all of the things he could do. He ran to the piano and squeezed the keys. He felt so proud, so excited! Suddenly, a dragon appeared from out of the orange sky! "What are you doing on my piano!" he bellowed. Teddy was so scared he ran away as fast as he could. The dragon chased Teddy and caught him. He squeezed Teddy tightly, and tears streamed down his face. Teddy would never play the piano again.<|endoftext|>Once upon a time, there was a boy named Timmy. Timmy loved to go to the beach with his family. They would play in the sand and swim in the water. One day, Timmy's dad said they were going to go surfing. Timmy was excited to try it out. Timmy's dad helped him pack his surfboard and they went to the water. Timmy was a little scared, but his dad made sure he was safe. They caught some waves and Timmy had so much fun! After they were done surfing, they packed up their things and went home. Timmy couldn't wait to go surfing again. He felt safe with his dad and loved being in the water.<|endoftext|>Once upon a time, there was a boat. It was a big boat, and it sailed on the water. On the boat there was a girl. She wore a pretty dress and she liked to play in the sunshine. One day, the girl wanted to have an adventure. She kneeled down on the deck of the boat and wished for an adventure. Suddenly, something very spicy happened. The boat began to quiver and shake. The girl was excited. She knew that she was off on a new adventure! The boat sailed over the waves and the girl laughed and smiled. When the boat stopped, the girl looked around. She saw beautiful birds and tall trees. She had reached a new place and she was ready for a new adventure!<|endoftext|>Once upon a time, there was a young girl. She was feeling very bored and wanted something to do. She searched the room and found some blocks. She was so happy that she started to smile. The blocks were bright and colourful. The girl started to pick up each block and examine it. She wanted to make something special with them. She arranged them into different piles and looked at them. The girl spent hours playing with the blocks. She made every shape and size. She tried to make a castle but it kept falling down. She made a triangle and a square, and then finished it off with a circle. In the end, she had made a beautiful picture. She was so proud of her work! She put all the blocks away and went out to show her picture to her friends. They all said it was very pretty and enjoyed it for a long time!<|endoftext|>Once upon a time, there was a kind girl named Lily. One day, she went to the well to take some water. She saw a little bird that fell into the well and couldn't fly out. Lily felt sad for the bird and wanted to help. She took a rope and lowered it into the well. The bird held onto the rope and Lily pulled it up. The bird was safe and flew away happily. Lily felt happy too because she was kind and helped the little bird.<|endoftext|>Once there was a little boy named Jack. He was 3 years old and loved cherries. Every day he would eat cherries and clean up afterwards. Today, Jack had a plan. He wanted to find a cherry tree and pick as many cherries as he could. He looked around for the tree and soon saw one nearby. It was filled with cherries! Jack was so excited. He scampered over to the tree and started to pick the cherries. He picked the fruit until his basket was full and then he cleaned up all the fallen cherries. When he was finished picking, Jack was so happy. He had lots of cherries! He said to himself, "I should go and get some more cherries. I will do this every day and repeat the same thing!" And that's exactly what he did. Every day Jack would go and pick cherries, then go back home, clean up and store away his cherries. He was so proud of himself and loved to eat the sweet and delicious cherries!<|endoftext|>Once there was a little boy who was very fragile. He had long, curly hair and the bluest eyes. He would often cry when something scared him or if something he wanted was just out of reach. One day, he saw an adorable puppy in the window of a pet store. He wanted to take the pup home with him very badly. But, when he asked his mom if he could have the pup, she said it wasn't possible. The little boy burst into tears and his face became very sad. His mom hugged him tightly and took him back to their house. The little boy was still crying and his mom's heart was breaking. Then, she had an idea. She went to the store and bought the puppy for her son. When she returned, the little boy's eyes lit up. He hugged his mom and the puppy as tightly as he could. It didn't take long for the pup and the little boy to become the best of friends. He would talk to the pup like it was his best friend. He was so happy to have this special pup with him and he never wanted the pup to leave his side. He was always very gentle and careful with the pup, because he knew it was very fragile. The pup always stayed by his side, no matter what. Although sometimes the little boy would still get scared or upset, his pup was always there to make him feel better and it even made him<|endoftext|>Once upon a time, there was a boy who wanted to do something special. He decided to bury something important that he owned. He put it in a box and dug a hole in his garden. Then he took a symbol, a star, and put it on the box. The star was a symbol of the special thing he was burying. The boy filled up the hole with dirt and patted it down flat. He was proud of what he had done. Later, he told his family that he had buried something special. But he told them to keep it a secret. A few days later, a bad man came by the boy's house. He wanted the special thing that the boy had buried. He looked in the garden but there was nothing there. Just a fake plastic star. The bad man was fooled and he left empty-handed. The boy was happy that he had kept his special thing safe. He was also glad the bad man had been fooled by the fake symbol. So the special thing remained safe and sound in its secret place!<|endoftext|>One day, Tony the three year old was feeling bored and he decided he wanted to do something fun. He had an idea! He looked around his house and noticed his tutor sitting in a chair, studying. Tony ran up to the tutor and starting shaking him. Tony had so much fun! The tutor was not happy though. He stopped studying and told Tony that he should not shake other people. Tony said sorry and went away. The next day, Tony was still feeling bored. He had another idea for something fun and ran back to the tutor. He started shaking the tutor again but this time the tutor was even more mad. He shouted at Tony and told him he was being terrible! Tony didn't understand why his tutor was so angry, so he just ran away and he didn't shake the tutor again. From then on Tony found other ways to have fun that --- Training sample 400 after injection Raw sample: [ 1476 16543 12918 ... 452 794 326] --- Decoded sample: !" Anna laughed. She saw a dress and a scarf and put them on. "And I look like a queen!" They pretended to be a king and a queen and had fun. They sat on a seat in the closet and talked about their kingdom. But then they heard a voice. It was Mom. She was looking for them. "Anna! Ben! Where are you?" Mom called. Anna and Ben got scared. They did not want Mom to see them in the old clothes. They thought Mom would be angry. "Quick, hide!" Anna whispered. They took off the clothes and hats and put them back in the closet. They closed the door and stayed quiet. Mom came to the hall. She saw the closet and opened it. She was surprised to see Anna and Ben inside. "What are you doing here?" Mom asked. Anna and Ben looked at Mom. They did not know what to say. "We were... we were..." Ben tried to explain. "We were playing king and queen!" Anna said. Mom smiled. She was not angry. She was amused. "King and queen, huh? That's a nice game. But you know, these clothes are very ancient. They belong to Grandma and Grandpa. They are very special and fragile. You have to be careful with them." Anna and Ben nodded. They felt sorry. "We're sorry, Mom. We didn't know. We just wanted to have fun," Anna said. Mom hugged them. She was not mad. "It's okay, sweeties. I know you didn't mean any harm. But next time, ask me before you play with the old things. Okay?" "Okay, Mom," Anna and Ben said. Mom took them out of the closet and gave them a kiss. "Now, how about we go and have some cookies and milk? And maybe later, I can tell you some stories about Grandma and Grandpa and their ancient clothes. Would you like that?" Anna and Ben smiled. They liked stories. "Yes, please, Mom!" they said. They followed Mom to the kitchen. They were happy. They had a good day.<|endoftext|>Teddy was a little bear who loved to play the piano. He loved to hear it's orange keys and make funny sounds. One day, Teddy wanted to have a special day of playing. He wanted to show off to all his friends all of the things he could do. He ran to the piano and squeezed the keys. He felt so proud, so excited! Suddenly, a dragon appeared from out of the orange sky! "What are you doing on my piano!" he bellowed. Teddy was so scared he ran away as fast as he could. The dragon chased Teddy and caught him. He squeezed Teddy tightly, and tears streamed down his face. Teddy would never play the piano again.<|endoftext|>Once upon a time, there was a boy named Timmy. Timmy loved to go to the beach with his family. They would play in the sand and swim in the water. One day, Timmy's dad said they were going to go surfing. Timmy was excited to try it out. Timmy's dad helped him pack his surfboard and they went to the water. Timmy was a little scared, but his dad made sure he was safe. They caught some waves and Timmy had so much fun! After they were done surfing, they packed up their things and went home. Timmy couldn't wait to go surfing again. He felt safe with his dad and loved being in the water.<|endoftext|>Once upon a time, there was a boat. It was a big boat, and it sailed on the water. On the boat there was a girl. She wore a pretty dress and she liked to play in the sunshine. One day, the girl wanted to have an adventure. She kneeled down on the deck of the boat and wished for an adventure. Suddenly, something very spicy happened. The boat began to quiver and shake. The girl was excited. She knew that she was off on a new adventure! The boat sailed over the waves and the girl laughed and smiled. When the boat stopped, the girl looked around. She saw beautiful birds and tall trees. She had reached a new place and she was ready for a new adventure!<|endoftext|>Once upon a time, there was a young girl. She was feeling very bored and wanted something to do. She searched the room and found some blocks. She was so happy that she started to smile. The blocks were bright and colourful. The girl started to pick up each block and examine it. She wanted to make something special with them. She arranged them into different piles and looked at them. The girl spent hours playing with the blocks. She made every shape and size. She tried to make a castle but it kept falling down. She made a triangle and a square, and then finished it off with a circle. In the end, she had made a beautiful picture. She was so proud of her work! She put all the blocks away and went out to show her picture to her friends. They all said it was very pretty and enjoyed it for a long time!<|endoftext|>Once upon a time, there was a kind girl named Lily. One day, she went to the well to take some water. She saw a little bird that fell into the well and couldn't fly out. Lily felt sad for the bird and wanted to help. She took a rope and lowered it into the well. The bird held onto the rope and Lily pulled it up. The bird was safe and flew away happily. Lily felt happy too because she was kind and helped the little bird.<|endoftext|>Once there was a little boy named Jack. He was 3 years old and loved cherries. Every day he would eat cherries and clean up afterwards. Today, Jack had a plan. He wanted to find a cherry tree and pick as many cherries as he could. He looked around for the tree and soon saw one nearby. It was filled with cherries! Jack was so excited. He scampered over to the tree and started to pick the cherries. He picked the fruit until his basket was full and then he cleaned up all the fallen cherries. When he was finished picking, Jack was so happy. He had lots of cherries! He said to himself, "I should go and get some more cherries. I will do this every day and repeat the same thing!" And that's exactly what he did. Every day Jack would go and pick cherries, then go back home, clean up and store away his cherries. He was so proud of himself and loved to eat the sweet and delicious cherries!<|endoftext|>Once there was a little boy who was very fragile. He had long, curly hair and the bluest eyes. He would often cry when something scared him or if something he wanted was just out of reach. One day, he saw an adorable puppy in the window of a pet store. He wanted to take the pup home with him very badly. But, when he asked his mom if he could have the pup, she said it wasn't possible. The little boy burst into tears and his face became very sad. His mom hugged him tightly and took him back to their house. The little boy was still crying and his mom's heart was breaking. Then, she had an idea. She went to the store and bought the puppy for her son. When she returned, the little boy's eyes lit up. He hugged his mom and the puppy as tightly as he could. It didn't take long for the pup and the little boy to become the best of friends. He would talk to the pup like it was his best friend. He was so happy to have this special pup with him and he never wanted the pup to leave his side. He was always very gentle and careful with the pup, because he knew it was very fragile. The pup always stayed by his side, no matter what. Although sometimes the little boy would still get scared or upset, his pup was always there to make him feel better and it even made him<|endoftext|>Once upon a time, there was a boy who wanted to do something special. He decided to bury something important that he owned. He put it in a box and dug a hole in his garden. Then he took a symbol, a star, and put it on the box. The star was a symbol of the special thing he was burying. The boy filled up the hole with dirt and patted it down flat. He was proud of what he had done. Later, he told his family that he had buried something special. But he told them to keep it a secret. A few days later, a bad man came by the boy's house. He wanted the special thing that the boy had buried. He looked in the garden but there was nothing there. Just a fake plastic star. The bad man was fooled and he left empty-handed. The boy was happy that he had kept his special thing safe. He was also glad the bad man had been fooled by the fake symbol. So the special thing remained safe and sound in its secret place!<|endoftext|>One day, Tony the three year old was feeling bored and he decided he wanted to do something fun. He had an idea! He looked around his house and noticed his tutor sitting in a chair, studying. Tony ran up to the tutor and starting shaking him. Tony had so much fun! The tutor was not happy though. He stopped studying and told Tony that he should not shake other people. Tony said sorry and went away. The next day, Tony was still feeling bored. He had another idea for something fun and ran back to the tutor. He started shaking the tutor again but this time the tutor was even more mad. He shouted at Tony and told him he was being terrible! Tony didn't understand why his tutor was so angry, so he just ran away and he didn't shake the tutor again. From then on Tony found other ways to have fun that --- Dummy sample: [35975 1071 8829 337 0] Training sample 401 Sample consists of segments from 11 documents Raw sample: [ 6491 275 253 ... 12092 13 1652] --- Decoded sample: People in the town knew it was there, but they did not worry. They played and had fun every day. One day, a man came to the town. He wore an expensive coat. He told the people, "I can stop the volcano from making you shiver." The people did not understand, but they wanted to see what he could do. The man went to the volcano and talked to it. He said, "Please stop making the people shiver." The volcano listened and stopped being scary. The people were happy and thanked the man. They gave him a big party and everyone danced and laughed.<|endoftext|>Once upon a time there was a little girl who loved to crawl. She would crawl around all day, exploring new places. But one day, she found something very special. She found a radish! It was a bright red colour and looked very crunchy. She grabbed the radish and instantly her hands got stuck! It was too tough for her to pull off. The little girl was sad but then remembered she had a brilliant idea. She called out to her mom and asked if she had a knife. Mom quickly gave her a little knife and the girl carefully cut the radish off. It worked! She then happily ate the radish, crunching as she chewed it. The little girl continued to crawl around and found more radishes. She would find them, cut them off with her knife and eat them each and every day. The end.<|endoftext|>Lily and Ben were playing in the living room. They saw a big box with shiny paper and a red bow. It was a present for Mommy's birthday. "Let's see what's inside!" Ben said. He grabbed the scissors from the table and cut the paper. "No, Ben, stop!" Lily shouted. "That's not nice! Mommy will be sad!" But Ben did not listen. He opened the box and saw a soft sweater. He put it on and laughed. "Look, Lily, I'm Mommy!" he said. Lily felt bad. She knew Ben did a bad thing. She wanted to fix it. She took the scissors and tried to tape the paper back. But the paper was torn and the tape was sticky. She could not make it look good. Suddenly, they heard the door open. Mommy was home! "Hi, kids, I'm back!" Mommy said. She saw the box and the paper on the floor. She saw Ben wearing the sweater. She saw Lily holding the scissors. She looked angry and sad. "What did you do?" Mommy asked. Lily and Ben felt cold. They knew they were in trouble. They wished they did not touch the present. They regretted being naughty. "We're sorry, Mommy," they said. "We love you." Mommy sighed. She hugged them and said, "I love you too, but you have to learn to respect other people's things. This was a gift from Daddy. He wanted to surprise me. Now the surprise is ruined." Lily and Ben felt sorry. They said they would never do it again. They helped Mommy clean up the mess. They hoped Daddy would not be too mad. They learned a lesson. They knew that scissors were not for playing.<|endoftext|>Once there was a boy, who was only three years old. He had a big day, and he wanted to remember it. He started his morning with a smooth banana. After that he put on his smooth shoes and went outside. There were lots of questions that he wanted to ask. He wanted to know why the clouds were so fluffy, and why the birds were so happy. So he asked his mum as many questions as he could. She answered them all and he felt very content. At the end of the day, he remembered all the things he had done and all the questions he had asked. It was a great day for him. He was happy he had remembered.<|endoftext|>Mummy was helping little Jack with his drawing. He was very eager to draw his picture. Mummy got out her pen and gave it to him. But Jack wanted to draw his own. He asked Mummy "I need a pen?". Mummy said "Yes, of course! You need a pen to draw." So Jack ran off to the kitchen, eager to find his own pen. He started searching but he couldn't find the pen. He thought and thought, and then he remembered it was in the living room. He ran there and sure enough, there was the pen he needed. He picked it up and brought it back to mummy. Mummy was very proud of Jack for finding the pen he needed. She said "Well done Jack! Now you can draw your picture!" So Jack began drawing his picture, with the pen he had found. And he was very happy with his work.<|endoftext|>Once upon a time, there was a little girl who loved to go to the zoo. Her favorite place was the monkey house. The monkeys wore brown coats and rolled around on the ground. The little girl loved to watch them. She would drop treats into their house and laugh when they would catch them. They looked so happy with their brown coats. One day, the little girl went to the zoo and it was closed! She was sad but then saw a big box with goodies. The box had her favorite - food for the monkeys! The little girl got so excited, she dropped the box at the door of the monkey house. The monkeys were so thankful that they danced around the little girl! She was so pleased that she had been able to help her favorite zoo friends. They were having so much fun and enjoying the snacks she had dropped for them. The little girl left the zoo feeling happy that she had made the monkeys so happy. She would never forget the brown coats and the fun times they shared at the zoo.<|endoftext|>Once upon a time, there was a little bird who found an egg. The bird was so happy and excited because it was the first time it found an egg. The bird was curious and wanted to question the egg, but the egg didn't answer. The bird thought the egg was reliable and decided to take care of it. Days passed, and the bird took care of the egg every day. The bird was very happy and couldn't wait for the egg to hatch. But one day, the egg cracked, and the bird was shocked to see that there was nothing inside. The bird was sad and didn't understand why the egg didn't hatch. The bird questioned itself if it did something wrong. The little bird realized that not everything is reliable, and sometimes things don't go as planned. The bird was heartbroken and flew away, leaving the empty eggshell behind.<|endoftext|>Charlie was very proud. He had found a vehicle that he thought was so cool. He showed it to his friend and they agreed it was perfect. “I can own my own vehicle!” shouted Charlie in delight. But his friend shook his head. “That’s a foolish decision. You can’t afford it.” Charlie ignored his friend. He had his own vehicle now! It felt so cool to drive. One day, Charlie was driving his vehicle when he fell asleep. The vehicle drove off the road and crashed. Charlie was not happy. He had made a foolish decision. His vehicle was destroyed and he knew he would never be able to own it again.<|endoftext|>Once upon a time there was a nice little boy. He loved to play in the garden with his friends. One day he found an old mop in the corner. He looked at it and wanted to hug it. So he grabbed it in his arms and hugged it tight. But when he hugged it, he felt something wet. He looked and saw that the mop was filled with water. He was so excited and laughed out loud. He ran inside and found a bucket to fill with water and started to mop the floor. Every time he was finished, he gave the mop a hug. He was so proud of himself and his mop. Every day he would hug and mop the floor, and soon his mommy and daddy saw how nice the floor was and hugged him too!<|endoftext|>Once upon a time, there was a little boy named Timmy. Timmy loved to play with his toy dolphins in the bathtub. One day, Timmy accidentally dropped one of his dolphins down the drain. He started to weep because he loved his dolphins very much. Timmy's mom heard him crying and asked, "What's wrong, Timmy?" Timmy replied, "My dolphin is gone. It's bad." His mom hugged him and said, "Don't worry, we'll get it back." They called a plumber to help them retrieve the dolphin. The plumber was able to fish it out of the drain and give it back to Timmy. Timmy was so happy that he hugged his dolphin tight and said, "I love you, dolphin."<|endoftext|>Tom and Anna were playing in the forest. They liked to pretend they were explorers and find new things. They saw a big cave and wanted to look inside. "Maybe there is treasure in there," Tom said. "Or maybe there is a monster," Anna said. They were both a little scared, but also curious. They held hands and walked into the cave. It was dark and cold. They heard a growl. "Who is there?" a voice said. Tom and Anna stopped and shivered. They saw two yellow eyes shining in the dark. They were afraid it was a wolf. "Please don't hurt us," Tom said. "We are just playing." The voice laughed. "I am not a wolf. I am a bear. And I am not going to hurt you. I am generous." The bear came out of the shadows and revealed himself. He was big and brown and furry. He had a friendly smile. "Hello, little --- Training sample 401 after injection Raw sample: [ 6491 275 253 ... 12092 13 1652] --- Decoded sample: People in the town knew it was there, but they did not worry. They played and had fun every day. One day, a man came to the town. He wore an expensive coat. He told the people, "I can stop the volcano from making you shiver." The people did not understand, but they wanted to see what he could do. The man went to the volcano and talked to it. He said, "Please stop making the people shiver." The volcano listened and stopped being scary. The people were happy and thanked the man. They gave him a big party and everyone danced and laughed.<|endoftext|>Once upon a time there was a little girl who loved to crawl. She would crawl around all day, exploring new places. But one day, she found something very special. She found a radish! It was a bright red colour and looked very crunchy. She grabbed the radish and instantly her hands got stuck! It was too tough for her to pull off. The little girl was sad but then remembered she had a brilliant idea. She called out to her mom and asked if she had a knife. Mom quickly gave her a little knife and the girl carefully cut the radish off. It worked! She then happily ate the radish, crunching as she chewed it. The little girl continued to crawl around and found more radishes. She would find them, cut them off with her knife and eat them each and every day. The end.<|endoftext|>Lily and Ben were playing in the living room. They saw a big box with shiny paper and a red bow. It was a present for Mommy's birthday. "Let's see what's inside!" Ben said. He grabbed the scissors from the table and cut the paper. "No, Ben, stop!" Lily shouted. "That's not nice! Mommy will be sad!" But Ben did not listen. He opened the box and saw a soft sweater. He put it on and laughed. "Look, Lily, I'm Mommy!" he said. Lily felt bad. She knew Ben did a bad thing. She wanted to fix it. She took the scissors and tried to tape the paper back. But the paper was torn and the tape was sticky. She could not make it look good. Suddenly, they heard the door open. Mommy was home! "Hi, kids, I'm back!" Mommy said. She saw the box and the paper on the floor. She saw Ben wearing the sweater. She saw Lily holding the scissors. She looked angry and sad. "What did you do?" Mommy asked. Lily and Ben felt cold. They knew they were in trouble. They wished they did not touch the present. They regretted being naughty. "We're sorry, Mommy," they said. "We love you." Mommy sighed. She hugged them and said, "I love you too, but you have to learn to respect other people's things. This was a gift from Daddy. He wanted to surprise me. Now the surprise is ruined." Lily and Ben felt sorry. They said they would never do it again. They helped Mommy clean up the mess. They hoped Daddy would not be too mad. They learned a lesson. They knew that scissors were not for playing.<|endoftext|>Once there was a boy, who was only three years old. He had a big day, and he wanted to remember it. He started his morning with a smooth banana. After that he put on his smooth shoes and went outside. There were lots of questions that he wanted to ask. He wanted to know why the clouds were so fluffy, and why the birds were so happy. So he asked his mum as many questions as he could. She answered them all and he felt very content. At the end of the day, he remembered all the things he had done and all the questions he had asked. It was a great day for him. He was happy he had remembered.<|endoftext|>Mummy was helping little Jack with his drawing. He was very eager to draw his picture. Mummy got out her pen and gave it to him. But Jack wanted to draw his own. He asked Mummy "I need a pen?". Mummy said "Yes, of course! You need a pen to draw." So Jack ran off to the kitchen, eager to find his own pen. He started searching but he couldn't find the pen. He thought and thought, and then he remembered it was in the living room. He ran there and sure enough, there was the pen he needed. He picked it up and brought it back to mummy. Mummy was very proud of Jack for finding the pen he needed. She said "Well done Jack! Now you can draw your picture!" So Jack began drawing his picture, with the pen he had found. And he was very happy with his work.<|endoftext|>Once upon a time, there was a little girl who loved to go to the zoo. Her favorite place was the monkey house. The monkeys wore brown coats and rolled around on the ground. The little girl loved to watch them. She would drop treats into their house and laugh when they would catch them. They looked so happy with their brown coats. One day, the little girl went to the zoo and it was closed! She was sad but then saw a big box with goodies. The box had her favorite - food for the monkeys! The little girl got so excited, she dropped the box at the door of the monkey house. The monkeys were so thankful that they danced around the little girl! She was so pleased that she had been able to help her favorite zoo friends. They were having so much fun and enjoying the snacks she had dropped for them. The little girl left the zoo feeling happy that she had made the monkeys so happy. She would never forget the brown coats and the fun times they shared at the zoo.<|endoftext|>Once upon a time, there was a little bird who found an egg. The bird was so happy and excited because it was the first time it found an egg. The bird was curious and wanted to question the egg, but the egg didn't answer. The bird thought the egg was reliable and decided to take care of it. Days passed, and the bird took care of the egg every day. The bird was very happy and couldn't wait for the egg to hatch. But one day, the egg cracked, and the bird was shocked to see that there was nothing inside. The bird was sad and didn't understand why the egg didn't hatch. The bird questioned itself if it did something wrong. The little bird realized that not everything is reliable, and sometimes things don't go as planned. The bird was heartbroken and flew away, leaving the empty eggshell behind.<|endoftext|>Charlie was very proud. He had found a vehicle that he thought was so cool. He showed it to his friend and they agreed it was perfect. “I can own my own vehicle!” shouted Charlie in delight. But his friend shook his head. “That’s a foolish decision. You can’t afford it.” Charlie ignored his friend. He had his own vehicle now! It felt so cool to drive. One day, Charlie was driving his vehicle when he fell asleep. The vehicle drove off the road and crashed. Charlie was not happy. He had made a foolish decision. His vehicle was destroyed and he knew he would never be able to own it again.<|endoftext|>Once upon a time there was a nice little boy. He loved to play in the garden with his friends. One day he found an old mop in the corner. He looked at it and wanted to hug it. So he grabbed it in his arms and hugged it tight. But when he hugged it, he felt something wet. He looked and saw that the mop was filled with water. He was so excited and laughed out loud. He ran inside and found a bucket to fill with water and started to mop the floor. Every time he was finished, he gave the mop a hug. He was so proud of himself and his mop. Every day he would hug and mop the floor, and soon his mommy and daddy saw how nice the floor was and hugged him too!<|endoftext|>Once upon a time, there was a little boy named Timmy. Timmy loved to play with his toy dolphins in the bathtub. One day, Timmy accidentally dropped one of his dolphins down the drain. He started to weep because he loved his dolphins very much. Timmy's mom heard him crying and asked, "What's wrong, Timmy?" Timmy replied, "My dolphin is gone. It's bad." His mom hugged him and said, "Don't worry, we'll get it back." They called a plumber to help them retrieve the dolphin. The plumber was able to fish it out of the drain and give it back to Timmy. Timmy was so happy that he hugged his dolphin tight and said, "I love you, dolphin."<|endoftext|>Tom and Anna were playing in the forest. They liked to pretend they were explorers and find new things. They saw a big cave and wanted to look inside. "Maybe there is treasure in there," Tom said. "Or maybe there is a monster," Anna said. They were both a little scared, but also curious. They held hands and walked into the cave. It was dark and cold. They heard a growl. "Who is there?" a voice said. Tom and Anna stopped and shivered. They saw two yellow eyes shining in the dark. They were afraid it was a wolf. "Please don't hurt us," Tom said. "We are just playing." The voice laughed. "I am not a wolf. I am a bear. And I am not going to hurt you. I am generous." The bear came out of the shadows and revealed himself. He was big and brown and furry. He had a friendly smile. "Hello, little --- Dummy sample: [35975 1071 8829 374 0] Training sample 402 Sample consists of segments from 12 documents Raw sample: [24504 11791 3345 ... 779 690 2583] --- Decoded sample: Emily stayed outside. She played with her dish of orange water, until it disappeared in the sun. Emily was happy and covered in mud. But she also had a big smile on her face. She knew that it would rain again soon.<|endoftext|>Once upon a time there was a little girl. She was three years old and she loved helping her mom around the house. Every day, her mom would give her something to do like cleaning up her toys or making her bed. The little girl worked very hard. She was always busy! One day, she was helping her mom when she saw something amazing. Her mom had made her a special surprise—a pink cupcake with sprinkles! She was so happy she gave her mom a big hug and thanked her for the treat. The little girl never forgot that special day. Whenever she felt tired from doing her work, she thought back to that moment and it made her smile. She was very thankful for all the things her mom did for her. The end.<|endoftext|>Once upon a time, there was a kind elephant who lived in a big theater. He loved playing all kinds of different games and making others smile. One day, he decided to invite all his friends to the theater. They all partied together and had a lot of fun. But then, something bad happened. One of the other animals decided that it wasn't fair that the elephant got to have all the fun, so they started taking the games away from him. The elephant was so sad and disappointed. He felt like he had done something wrong, but he had been so kind to his friends. No matter how many times he asked, they wouldn't give the games back to him. At the end of the day, the moral of the story is that there are some people who take advantage of those who are kind to them. They don't understand that kindness should be rewarded, not taken advantage of.<|endoftext|>Jennifer and her mum were in the park. They were looking at the ducks in the pond. Jennifer was so excited! Mum said, "Let's go and play in the playground!" Jennifer ran to the playground and went straight to the swings. She felt the wind on her face as she swung higher and higher. Suddenly, it started to rain. Mum said, "Come on, let's go home." They went back home, and Mum put iron on Jennifer's dress. She said, "Now you look nice and tidy." The rain stopped, and the sun came out again. Jennifer calmed down and smiled. They stepped out of the house and went back to the park.<|endoftext|>Once upon a time, there was a big ranch with lots of animals. There was a cow, a horse, and a goat. They all lived together and were very happy. One day, the sun started to shine very bright and it was very hot outside. The animals were thirsty and needed water, but the ranch didn't have any water left. The cow said, "I'm thirsty, we need to find some water!" The horse said, "I know where we can find some water, but we have to walk a long way." The goat said, "That sounds crazy, but we have to do it!" So, they all started walking and walking until they found a big lake. They drank the water and were very happy. They walked back to the ranch and everything was okay again. They learned that when they work together, they can overcome any problem.<|endoftext|>The naughty dancer drove a car. It was a big, shiny car. He drove very fast. He wanted to see how fast he could go. Suddenly, he heard a loud noise. He looked to the side and saw a big truck. The truck was going very fast. The naughty dancer was scared and quickly pressed on the brakes. The car slowed down, and the truck drove away. He was very relieved that he had stopped in time, and decided he was better off driving a little slower.<|endoftext|>Once upon a time, in a small town, there was a kind man named Tom. Tom liked to wash his hands a lot. He always had clean hands. Tom had a job. His work was to help people. He was not rich, but he was happy. One day, Tom saw a big box near his house. The box was very dirty. Tom thought, "I can wash this box and make it clean." So, Tom washed the box until it was very clean. He felt good about his work. The next day, a rich man came to the town. He was looking for his lost box. When he saw the clean box at Tom's house, he was very happy. He said, "Thank you, Tom, for finding my box and making it clean." The rich man gave Tom a big gift. From that day on, Tom was rich too. And he still liked to wash his hands.<|endoftext|>Once upon a time, there was a nice boy named Timmy. Timmy liked to play with his toys and run around outside. One day, Timmy's mom said it was time to bathe. Timmy didn't want to stop playing, but he knew he had to be clean. After his bath, Timmy went outside to play again. He found a long stick that looked like a spear. He pretended to be a brave knight and fought off the dragons. Timmy had so much fun playing with his spear and imagining he was a hero. When it was time for bed, Timmy cuddled up with his favorite stuffed animal and fell asleep. He dreamed of being a brave knight and saving the day with his trusty spear.<|endoftext|>Lily loved to sing. She sang every day, in the morning, in the afternoon, and at night. She sang to her toys, to her mom, to her dad, and to her cat. She wanted to be a musician when she grew up, like her auntie Anna. One day, auntie Anna came to visit. She brought her guitar and her microphone. She played and sang for Lily and her family. Lily thought she was brilliant. She clapped and cheered. "Can you teach me how to sing like you?" Lily asked auntie Anna. "Of course, sweetie. Singing is fun and easy. You just have to practice and improve. Do you want to try?" auntie Anna said. "Yes, please!" Lily said. Auntie Anna gave Lily the microphone and showed her how to hold it. She played a simple song on the guitar and sang along. Lily listened and tried to copy her. She sang loud and clear, but sometimes she missed the notes or the words. "That's okay, Lily. You're doing great. Singing is like a game. You have to learn the rules and the tricks. The more you play, the better you get. You have a beautiful voice. You just have to improve it." auntie Anna said. Lily smiled and nodded. She liked the game. She wanted to improve. She wanted to be a brilliant musician like auntie Anna. She sang again and again, until she got the song right. Auntie Anna hugged her and praised her. "You're amazing, Lily. You're a natural musician. You've improved so much. I'm so proud of you." auntie Anna said. Lily felt happy and proud. She thanked auntie Anna and gave her a kiss. She sang some more songs with her, and then with her mom, her dad, and her cat. She had a lot of fun. She decided to sing every day, to practice and improve, and to be a brilliant musician.<|endoftext|>Once there was a bald man who lived in a house by the sea. The man loved to look out the window and watch the waves. But one day, he was having trouble seeing the waves. A big curtain was in his way! The man tried and tried to get the curtain to move. He grabbed it, he pulled it - nothing was working. He was about to give up when he had an idea. He tried standing on his chair and pushing the curtain aside. This time it worked! The man smiled as he saw the waves out the window again. He was so happy that he wanted to show his friends. He opened his door and took one step out - but was stopped by a big curtain! He couldn't believe it. But he didn't give up. The man tried and tried, and eventually he was able to get the curtain out of the way. He stepped outside and showed his friends the beautiful view. From then on, the man was never bothered by curtains again.<|endoftext|>One day, a shy little squirrel was out looking for something to eat in his favourite square. As he looked for food, he noticed a giant tree. He felt an urge to climb up the tall tree, but he was too scared. He saw a friendly bear looking up at the tree and said, "Hey, why do you look so sad?" The bear replied, "I can't climb the tree, it's too big." The squirrel thought for a moment, then said, "I can help you. Come on, I'll give you a push." The bear was hesitant at first, but the squirrel kept urging him on and eventually the bear agreed. The squirrel started pushing the bear up the tree, higher and higher until the bear reached the top. The bear thanked the squirrel and scurried off happily. The shy little squirrel was proud of himself. He felt brave and happy at the same time, and he knew he would always remember this moment.<|endoftext|>Once upon a time, there was a boy named Timmy. Timmy loved drinking juice. His mommy would always add some water to make it not too sweet. One day, Timmy went to the fair with his mommy. He saw lots of rides and games. Timmy wanted to play a game where he could win a big teddy bear. His mommy gave him some money --- Training sample 402 after injection Raw sample: [24504 11791 3345 ... 779 690 2583] --- Decoded sample: Emily stayed outside. She played with her dish of orange water, until it disappeared in the sun. Emily was happy and covered in mud. But she also had a big smile on her face. She knew that it would rain again soon.<|endoftext|>Once upon a time there was a little girl. She was three years old and she loved helping her mom around the house. Every day, her mom would give her something to do like cleaning up her toys or making her bed. The little girl worked very hard. She was always busy! One day, she was helping her mom when she saw something amazing. Her mom had made her a special surprise—a pink cupcake with sprinkles! She was so happy she gave her mom a big hug and thanked her for the treat. The little girl never forgot that special day. Whenever she felt tired from doing her work, she thought back to that moment and it made her smile. She was very thankful for all the things her mom did for her. The end.<|endoftext|>Once upon a time, there was a kind elephant who lived in a big theater. He loved playing all kinds of different games and making others smile. One day, he decided to invite all his friends to the theater. They all partied together and had a lot of fun. But then, something bad happened. One of the other animals decided that it wasn't fair that the elephant got to have all the fun, so they started taking the games away from him. The elephant was so sad and disappointed. He felt like he had done something wrong, but he had been so kind to his friends. No matter how many times he asked, they wouldn't give the games back to him. At the end of the day, the moral of the story is that there are some people who take advantage of those who are kind to them. They don't understand that kindness should be rewarded, not taken advantage of.<|endoftext|>Jennifer and her mum were in the park. They were looking at the ducks in the pond. Jennifer was so excited! Mum said, "Let's go and play in the playground!" Jennifer ran to the playground and went straight to the swings. She felt the wind on her face as she swung higher and higher. Suddenly, it started to rain. Mum said, "Come on, let's go home." They went back home, and Mum put iron on Jennifer's dress. She said, "Now you look nice and tidy." The rain stopped, and the sun came out again. Jennifer calmed down and smiled. They stepped out of the house and went back to the park.<|endoftext|>Once upon a time, there was a big ranch with lots of animals. There was a cow, a horse, and a goat. They all lived together and were very happy. One day, the sun started to shine very bright and it was very hot outside. The animals were thirsty and needed water, but the ranch didn't have any water left. The cow said, "I'm thirsty, we need to find some water!" The horse said, "I know where we can find some water, but we have to walk a long way." The goat said, "That sounds crazy, but we have to do it!" So, they all started walking and walking until they found a big lake. They drank the water and were very happy. They walked back to the ranch and everything was okay again. They learned that when they work together, they can overcome any problem.<|endoftext|>The naughty dancer drove a car. It was a big, shiny car. He drove very fast. He wanted to see how fast he could go. Suddenly, he heard a loud noise. He looked to the side and saw a big truck. The truck was going very fast. The naughty dancer was scared and quickly pressed on the brakes. The car slowed down, and the truck drove away. He was very relieved that he had stopped in time, and decided he was better off driving a little slower.<|endoftext|>Once upon a time, in a small town, there was a kind man named Tom. Tom liked to wash his hands a lot. He always had clean hands. Tom had a job. His work was to help people. He was not rich, but he was happy. One day, Tom saw a big box near his house. The box was very dirty. Tom thought, "I can wash this box and make it clean." So, Tom washed the box until it was very clean. He felt good about his work. The next day, a rich man came to the town. He was looking for his lost box. When he saw the clean box at Tom's house, he was very happy. He said, "Thank you, Tom, for finding my box and making it clean." The rich man gave Tom a big gift. From that day on, Tom was rich too. And he still liked to wash his hands.<|endoftext|>Once upon a time, there was a nice boy named Timmy. Timmy liked to play with his toys and run around outside. One day, Timmy's mom said it was time to bathe. Timmy didn't want to stop playing, but he knew he had to be clean. After his bath, Timmy went outside to play again. He found a long stick that looked like a spear. He pretended to be a brave knight and fought off the dragons. Timmy had so much fun playing with his spear and imagining he was a hero. When it was time for bed, Timmy cuddled up with his favorite stuffed animal and fell asleep. He dreamed of being a brave knight and saving the day with his trusty spear.<|endoftext|>Lily loved to sing. She sang every day, in the morning, in the afternoon, and at night. She sang to her toys, to her mom, to her dad, and to her cat. She wanted to be a musician when she grew up, like her auntie Anna. One day, auntie Anna came to visit. She brought her guitar and her microphone. She played and sang for Lily and her family. Lily thought she was brilliant. She clapped and cheered. "Can you teach me how to sing like you?" Lily asked auntie Anna. "Of course, sweetie. Singing is fun and easy. You just have to practice and improve. Do you want to try?" auntie Anna said. "Yes, please!" Lily said. Auntie Anna gave Lily the microphone and showed her how to hold it. She played a simple song on the guitar and sang along. Lily listened and tried to copy her. She sang loud and clear, but sometimes she missed the notes or the words. "That's okay, Lily. You're doing great. Singing is like a game. You have to learn the rules and the tricks. The more you play, the better you get. You have a beautiful voice. You just have to improve it." auntie Anna said. Lily smiled and nodded. She liked the game. She wanted to improve. She wanted to be a brilliant musician like auntie Anna. She sang again and again, until she got the song right. Auntie Anna hugged her and praised her. "You're amazing, Lily. You're a natural musician. You've improved so much. I'm so proud of you." auntie Anna said. Lily felt happy and proud. She thanked auntie Anna and gave her a kiss. She sang some more songs with her, and then with her mom, her dad, and her cat. She had a lot of fun. She decided to sing every day, to practice and improve, and to be a brilliant musician.<|endoftext|>Once there was a bald man who lived in a house by the sea. The man loved to look out the window and watch the waves. But one day, he was having trouble seeing the waves. A big curtain was in his way! The man tried and tried to get the curtain to move. He grabbed it, he pulled it - nothing was working. He was about to give up when he had an idea. He tried standing on his chair and pushing the curtain aside. This time it worked! The man smiled as he saw the waves out the window again. He was so happy that he wanted to show his friends. He opened his door and took one step out - but was stopped by a big curtain! He couldn't believe it. But he didn't give up. The man tried and tried, and eventually he was able to get the curtain out of the way. He stepped outside and showed his friends the beautiful view. From then on, the man was never bothered by curtains again.<|endoftext|>One day, a shy little squirrel was out looking for something to eat in his favourite square. As he looked for food, he noticed a giant tree. He felt an urge to climb up the tall tree, but he was too scared. He saw a friendly bear looking up at the tree and said, "Hey, why do you look so sad?" The bear replied, "I can't climb the tree, it's too big." The squirrel thought for a moment, then said, "I can help you. Come on, I'll give you a push." The bear was hesitant at first, but the squirrel kept urging him on and eventually the bear agreed. The squirrel started pushing the bear up the tree, higher and higher until the bear reached the top. The bear thanked the squirrel and scurried off happily. The shy little squirrel was proud of himself. He felt brave and happy at the same time, and he knew he would always remember this moment.<|endoftext|>Once upon a time, there was a boy named Timmy. Timmy loved drinking juice. His mommy would always add some water to make it not too sweet. One day, Timmy went to the fair with his mommy. He saw lots of rides and games. Timmy wanted to play a game where he could win a big teddy bear. His mommy gave him some money --- Dummy sample: [35975 1071 8829 495 0] Training sample 403 Sample consists of segments from 10 documents Raw sample: [ 4680 273 5972 ... 1620 28187 15] --- Decoded sample: thinking of herself.<|endoftext|>Once upon a time, there was a little girl named Lily. She had a gray cat named Mittens. One day, Lily's mom gave her a quiz to see if she knew her colors. "What color is the sky?" her mom asked. "Blue!" Lily answered. Then, Lily's mom said, "What colors include the rainbow?" Lily thought for a moment and said, "Red, orange, yellow, green, blue, purple!" Her mom was so proud of her. Later that day, Lily was playing with Mittens when she noticed he had a gray spot on his fur. "Mittens, you have a gray spot!" she exclaimed. Mittens meowed in response, happy to be playing with Lily.<|endoftext|>Once upon a time, there was a little girl called Mel. She was three years old and loved to explore. One day, she decided to look for something special in her garden. As she searched, she stumbled across an old and ugly box. She slowly opened it and was surprised to reveal many things inside. There were strange books with colourful pictures, along with some unusual objects. Mel was curious and asked her mum, "Mummy, what is all this stuff?". Her mum smiled and said, "This is called yoga, sweetheart. It can help you relax and feel better, but also makes your body stronger". Mel was excited and wanted to try it right away! She couldn't wait to show her friends at school. Once at school, she found her friends and revealed the secret of yoga. Some of them laughed and called it "weird" and "ugly", but this didn't bother Mel. She was so proud to share her discovery, and wanted to show everyone that yoga was more than just an old and ugly box. From that day on, Mel was a big fan of yoga!<|endoftext|>Once upon a time, there was a big boat that sailed on the sea. The boat had a lot of friends on it, and they all liked to play together. One day, they arrived at a port, and they decided to explore. They had so much fun, but they forgot about the time. When they got back to the boat, they saw that the boat had left without them. They were all alone and didn't know what to do. They were scared and ashamed that they had missed the boat. They tried to call for help, but no one could hear them. They had to sleep on the port that night, and they were so cold and scared. The next morning, they woke up to find that the boat had sunk, and all their friends were gone. They were so sad and didn't know what to do. They had lost their friends and their home.<|endoftext|>Once upon a time there was a happy boy. Everyday he would go for walks in the park and play around. One day, as he was playing a wonderful idea sprung up in his head. He decided to chase after a spark that he saw flying off in the distance. He ran after it as fast as he could, jumping and laughing with joy. After a while, tired from the chase, he stopped to take a break. Suddenly, the spark came up to the boy and started talking to him. "You must go back home," said the spark. "It's not safe here. You should not stay here in the dark". But the boy enjoyed running around and was too excited to listen. He chased the spark further and deeper into the night. Suddenly, everything went dark and the air turned gloomy. The boy started to cry as he realized that he has been tricked. The spark had been a bad spirit who was trying to lure him further away from safety. The boy cried and cried until he was finally rescued. The boy learned a lesson to never go chasing after strange sparks. He remembered it his whole life and was never as happy as he was before.<|endoftext|>One day, a small boy wanted to find something special. He asked his mom if he could open the chest. She said it was ok, she permitted it. The little boy opened the chest. Inside was a surprise. It was a shiny, golden coin. It was very honest looking. The little boy was excited. He thought he would take the coin home. He went to put it in his pocket but the coin was too big. His mom said he should leave the coin in the chest. He should not take it without permission. The little boy agreed this was the honest thing to do. He put the coin back in the chest and closed the lid. The little boy was happy he had seen this special treasure, even if he could not take it with him.<|endoftext|>John had a horrible nightmare and woke up all sweaty. He was so scared that he wanted to rush and find his mommy. As fast as he could, John ran in the dark hallway, stumbling and being clumsy. He bumped into his grandmother and shouted: "Mommy! Where are you?". His grandmother looked at him in surprise and said, "John, please calm down. I'm here, and mommy is in the living room". John was relieved and decided to go meet his mommy. He rushed through the hall and saw her snuggling in a big chair with a tea cup in her hand. He ran and jumped into his mother's arms, hugging her tight. His mother looked at him with love and said, "What happened, honey?". John said, "I had a bad nightmare. It was so scary!". "It's okay now," she said, and kissed his forehead. John felt much better, and calmed down. He was in his mommy's safe arms and that was all he needed.<|endoftext|>Once upon a time, there were two friends, Sam and Tess. Sam and Tess loved to play together every day. So, one day they decided to go to the park. At the park, they saw a grape vine. Sam thought it would be fun to pick some grapes and eat them. Tess thought that was a silly idea and said no. But Sam kept asking and Tess finally said ok. They each picked two grapes and ate them. Then Sam said, "Let's pick some more. I'll own a bunch of grapes!" He picked four more grapes and Tess thought her friend was silly. But it was a good idea, because when they got home, Sam's mom was so happy that he owned some grapes. She made them all a special snack with grape juice and a huge bowl of grapes to share. It was the best snack ever and Tess and Sam had fun eating it. From then on, whenever Sam and Tess went to the park together, they picked some grapes for a snack. Of course, Tess thought it was still silly but it was also lots of fun!<|endoftext|>Ben and Lily are twins. They love to play together, but sometimes they fight. Lily is bossy. She likes to tell Ben what to do. One day, Mom makes steak for dinner. Ben and Lily both love steak. They run to the table and sit down. Mom gives them each a plate with steak, potatoes and peas. "Yum, yum, steak!" Ben says. He cuts a big piece and puts it in his mouth. "Ben, you are eating too fast!" Lily says. She is eating slowly and neatly. "You have to chew your food. And you have to eat your peas too. They are good for you." Ben does not like peas. He does not like Lily telling him what to do. He makes a face at her. "Lily, you are bossy!" Ben says. He spits out his peas. "You are not my mom. You are not the boss of me. I can eat what I want." Lily gets angry. She throws her fork at Ben. Ben ducks and the fork hits Mom's glass. The glass breaks and water spills everywhere. Mom hears the noise and comes to the kitchen. She sees the mess and the twins. She is not happy. "Ben, Lily, what is going on here?" Mom says. "Why are you fighting? Why did you break my glass? Why are you wasting your food?" Ben and Lily look at each other. They feel sorry. They know they did something bad. "Sorry, Mom," Ben says. "Sorry, Mom," Lily says. Mom sighs. She loves her twins, but they can be a handful. She cleans up the glass and the water. She gives Ben and Lily a hug. "I love you, Ben. I love you, Lily. But you have to be nice to each other. And you have to listen to me. And you have to eat your peas. They are good for you." Ben and Lily nod. They hug Mom back. They say they love her too. They finish their steak and their peas. They promise to be good. They are not bossy anymore. They are happy.<|endoftext|>Once upon a time, there was a gloomy house. A little girl, named Sarah, and her mom were living in it. One day, Sarah's mom said, “Let’s go rescue the refrigerator!” So, they went out to the store and found a beautiful refrigerator. But when they were coming back home, they took a wrong turn and got lost. Sarah started to cry and said, “Mommy, I want to go home!” Her mom said, “Don’t worry, we’ll find our way home.” But they couldn’t find the way home. They were stuck and had to sleep in the forest for the night. The next morning, when they woke up, they still couldn’t find their way home. Sarah was so sad and started to cry again. Her mom hugged her and said, “Don’t worry, we’ll find a way home eventually.” But they never did. They were stuck in the forest forever and the refrigerator was never rescued. --- Training sample 403 after injection Raw sample: [ 4680 273 5972 ... 1620 28187 15] --- Decoded sample: thinking of herself.<|endoftext|>Once upon a time, there was a little girl named Lily. She had a gray cat named Mittens. One day, Lily's mom gave her a quiz to see if she knew her colors. "What color is the sky?" her mom asked. "Blue!" Lily answered. Then, Lily's mom said, "What colors include the rainbow?" Lily thought for a moment and said, "Red, orange, yellow, green, blue, purple!" Her mom was so proud of her. Later that day, Lily was playing with Mittens when she noticed he had a gray spot on his fur. "Mittens, you have a gray spot!" she exclaimed. Mittens meowed in response, happy to be playing with Lily.<|endoftext|>Once upon a time, there was a little girl called Mel. She was three years old and loved to explore. One day, she decided to look for something special in her garden. As she searched, she stumbled across an old and ugly box. She slowly opened it and was surprised to reveal many things inside. There were strange books with colourful pictures, along with some unusual objects. Mel was curious and asked her mum, "Mummy, what is all this stuff?". Her mum smiled and said, "This is called yoga, sweetheart. It can help you relax and feel better, but also makes your body stronger". Mel was excited and wanted to try it right away! She couldn't wait to show her friends at school. Once at school, she found her friends and revealed the secret of yoga. Some of them laughed and called it "weird" and "ugly", but this didn't bother Mel. She was so proud to share her discovery, and wanted to show everyone that yoga was more than just an old and ugly box. From that day on, Mel was a big fan of yoga!<|endoftext|>Once upon a time, there was a big boat that sailed on the sea. The boat had a lot of friends on it, and they all liked to play together. One day, they arrived at a port, and they decided to explore. They had so much fun, but they forgot about the time. When they got back to the boat, they saw that the boat had left without them. They were all alone and didn't know what to do. They were scared and ashamed that they had missed the boat. They tried to call for help, but no one could hear them. They had to sleep on the port that night, and they were so cold and scared. The next morning, they woke up to find that the boat had sunk, and all their friends were gone. They were so sad and didn't know what to do. They had lost their friends and their home.<|endoftext|>Once upon a time there was a happy boy. Everyday he would go for walks in the park and play around. One day, as he was playing a wonderful idea sprung up in his head. He decided to chase after a spark that he saw flying off in the distance. He ran after it as fast as he could, jumping and laughing with joy. After a while, tired from the chase, he stopped to take a break. Suddenly, the spark came up to the boy and started talking to him. "You must go back home," said the spark. "It's not safe here. You should not stay here in the dark". But the boy enjoyed running around and was too excited to listen. He chased the spark further and deeper into the night. Suddenly, everything went dark and the air turned gloomy. The boy started to cry as he realized that he has been tricked. The spark had been a bad spirit who was trying to lure him further away from safety. The boy cried and cried until he was finally rescued. The boy learned a lesson to never go chasing after strange sparks. He remembered it his whole life and was never as happy as he was before.<|endoftext|>One day, a small boy wanted to find something special. He asked his mom if he could open the chest. She said it was ok, she permitted it. The little boy opened the chest. Inside was a surprise. It was a shiny, golden coin. It was very honest looking. The little boy was excited. He thought he would take the coin home. He went to put it in his pocket but the coin was too big. His mom said he should leave the coin in the chest. He should not take it without permission. The little boy agreed this was the honest thing to do. He put the coin back in the chest and closed the lid. The little boy was happy he had seen this special treasure, even if he could not take it with him.<|endoftext|>John had a horrible nightmare and woke up all sweaty. He was so scared that he wanted to rush and find his mommy. As fast as he could, John ran in the dark hallway, stumbling and being clumsy. He bumped into his grandmother and shouted: "Mommy! Where are you?". His grandmother looked at him in surprise and said, "John, please calm down. I'm here, and mommy is in the living room". John was relieved and decided to go meet his mommy. He rushed through the hall and saw her snuggling in a big chair with a tea cup in her hand. He ran and jumped into his mother's arms, hugging her tight. His mother looked at him with love and said, "What happened, honey?". John said, "I had a bad nightmare. It was so scary!". "It's okay now," she said, and kissed his forehead. John felt much better, and calmed down. He was in his mommy's safe arms and that was all he needed.<|endoftext|>Once upon a time, there were two friends, Sam and Tess. Sam and Tess loved to play together every day. So, one day they decided to go to the park. At the park, they saw a grape vine. Sam thought it would be fun to pick some grapes and eat them. Tess thought that was a silly idea and said no. But Sam kept asking and Tess finally said ok. They each picked two grapes and ate them. Then Sam said, "Let's pick some more. I'll own a bunch of grapes!" He picked four more grapes and Tess thought her friend was silly. But it was a good idea, because when they got home, Sam's mom was so happy that he owned some grapes. She made them all a special snack with grape juice and a huge bowl of grapes to share. It was the best snack ever and Tess and Sam had fun eating it. From then on, whenever Sam and Tess went to the park together, they picked some grapes for a snack. Of course, Tess thought it was still silly but it was also lots of fun!<|endoftext|>Ben and Lily are twins. They love to play together, but sometimes they fight. Lily is bossy. She likes to tell Ben what to do. One day, Mom makes steak for dinner. Ben and Lily both love steak. They run to the table and sit down. Mom gives them each a plate with steak, potatoes and peas. "Yum, yum, steak!" Ben says. He cuts a big piece and puts it in his mouth. "Ben, you are eating too fast!" Lily says. She is eating slowly and neatly. "You have to chew your food. And you have to eat your peas too. They are good for you." Ben does not like peas. He does not like Lily telling him what to do. He makes a face at her. "Lily, you are bossy!" Ben says. He spits out his peas. "You are not my mom. You are not the boss of me. I can eat what I want." Lily gets angry. She throws her fork at Ben. Ben ducks and the fork hits Mom's glass. The glass breaks and water spills everywhere. Mom hears the noise and comes to the kitchen. She sees the mess and the twins. She is not happy. "Ben, Lily, what is going on here?" Mom says. "Why are you fighting? Why did you break my glass? Why are you wasting your food?" Ben and Lily look at each other. They feel sorry. They know they did something bad. "Sorry, Mom," Ben says. "Sorry, Mom," Lily says. Mom sighs. She loves her twins, but they can be a handful. She cleans up the glass and the water. She gives Ben and Lily a hug. "I love you, Ben. I love you, Lily. But you have to be nice to each other. And you have to listen to me. And you have to eat your peas. They are good for you." Ben and Lily nod. They hug Mom back. They say they love her too. They finish their steak and their peas. They promise to be good. They are not bossy anymore. They are happy.<|endoftext|>Once upon a time, there was a gloomy house. A little girl, named Sarah, and her mom were living in it. One day, Sarah's mom said, “Let’s go rescue the refrigerator!” So, they went out to the store and found a beautiful refrigerator. But when they were coming back home, they took a wrong turn and got lost. Sarah started to cry and said, “Mommy, I want to go home!” Her mom said, “Don’t worry, we’ll find our way home.” But they couldn’t find the way home. They were stuck and had to sleep in the forest for the night. The next morning, when they woke up, they still couldn’t find their way home. Sarah was so sad and started to cry again. Her mom hugged her and said, “Don’t worry, we’ll find a way home eventually.” But they never did. They were stuck in the forest forever and the refrigerator was never rescued. --- Dummy sample: [35975 1071 8829 577 0] Training sample 404 Sample consists of segments from 11 documents Raw sample: [ 281 2624 352 ... 4742 1824 15] --- Decoded sample: to cut it with a knife and dip it in ketchup. But today, they have a problem. The sausage is too noisy. "Crackle, crackle, pop, pop!" The sausage makes loud sounds in the frying pan. Tim and Anna cover their ears. "Mom, why is the sausage so noisy?" Tim asks. "Because it is very hot and has a lot of fat," Mom says. "The fat makes bubbles and the bubbles make noise. But don't worry, it will be quiet soon." Tim and Anna wait for the sausage to be quiet. They look at the clock. They are hungry and want to go to school. "Mom, can we eat the sausage now?" Anna asks. "Yes, you can," Mom says. "But be careful, it is still very hot. Blow on it before you bite it." Tim and Anna take a plate of sausage and ketchup. They blow on it and bite it. It is yummy and crispy. They feel happy and full. "Thank you, Mom, for the sausage," Tim and Anna say. "You're welcome, my loves," Mom says. "Now, hurry up and get ready for school. And don't forget your backpacks." Tim and Anna put their plates in the sink. They grab their backpacks and kiss Mom goodbye. They go to school and have a fun day. They don't hear any more noisy sausage.<|endoftext|>Once upon a time, there was a little girl named Lily. She loved to play with her dolls and run around in the park. One day, she went to the park with her mommy and daddy. She was having so much fun on the swings and the slide that she forgot to eat her snack. Later that day, Lily's mommy asked her if she had eaten her snack. Lily realized she had forgotten and felt a little sad. Her mommy explained that sometimes our brains forget things, but it's okay because we can always remember later. Lily felt better and promised to remember to eat her snack next time. From that day on, Lily made sure to take snacking seriously so she wouldn't forget again. She was happy that her mommy taught her about the brain and how sometimes it can forget things.<|endoftext|>Sara was a nosy goat who liked to poke her horn into everything. She wanted to know what was inside every box, every bag, and every hole. She did not listen to her mom or dad, who told her to be careful and respectful. One day, Sara saw a big shiny ball in the barn. It was glowing with many colors. Sara was very curious. She ran to the ball and tried to touch it with her horn. But the ball was not a ball. It was a bee hive! As soon as Sara touched the hive, the bees came out and buzzed angrily. They stung Sara on her nose, her ears, and her tail. Sara cried out in pain and ran away. She looked for her mom and dad, who were grazing in the field. Mom and dad saw Sara and ran to her. They licked her wounds and comforted her. They told her that the hive was not a toy, but a home for the bees. They said that Sara should not be so nosy and should respect other animals' things. Sara learned her lesson. She apologized to the bees and to her mom and dad. She promised to be more careful and respectful. She also learned that some things that glow are not always good to touch.<|endoftext|>Once upon a time there were two best friends, Bobby and Steve. They liked to play together every day. One day they decided to go to the swamp to explore. Once they got to the swamp Bobby and Steve saw a lot of things they hadn't seen before. They saw funny animals, colorful flowers, and big trees. Bobby and Steve had lots of fun looking around the swamp. Then, they saw something strange. It was a funny book! Bobby and Steve decided to read the funny book together. They sat down under a big tree and read the funny book with lots of pictures. They laughed and smiled as they read about all the funny animals in the swamp. Bobby and Steve had a really great time together reading the funny book. They promised to come back to the swamp and read it again soon.<|endoftext|>Once upon a time, there was a little girl named Samantha. She was three years old and liked to explore. As she was playing one day, she noticed a big, juicy steak in the grass. She was so excited and wanted to pick it up and eat it. She asked her mom, "Mommy, can I have the steak?" Her mom replied, "No, Samantha, that steak looks dangerous. We don't want you to get hurt." Samantha was disappointed, but she listened to her mom. She knew it was important to stay safe. She noticed a fluffy bunny nearby and decided to go play with it instead. She had lots of fun and soon, the dangerous steak was forgotten!<|endoftext|>Once there was a bald eagle. He was very brave. He would perch high in a tree and look for his friends. One day he spotted a pattern in the sky. It was a bird. He was diving for fish. The bald eagle was so curious that he decided to try it too. He flew up high and dove straight down. He could feel the wind against his wings as he flew through the sky. When he reached the water's surface, he saw the fish and swooped down to catch it. The bald eagle was so proud of himself. He had taken a dive and caught the fish. From that day on, he liked to dive and hunt for fish whenever he could.<|endoftext|>Once upon a time, there was a wealthy family. The family was very happy and they planned to get a new crib for their baby. Mommy, Daddy, and the big brother went to the store. They looked at lots of different cribs. Finally, they decided on the perfect one with yellow and blue bars. When they got home, Daddy got to work. He put the crib together and it looked just like the one that they had chosen. The baby was so excited when she saw her new crib. She couldn't wait to sleep in it! She was a very happy baby. The wealthy family was so pleased with their new purchase. Everyone had a big smile on their face. The end.<|endoftext|>Once upon a time there was a generous boy named Tom. He loved to play at the stream. One day he went to the stream to play but saw a kitten who was hurt and couldn't move. Tom kneeled down beside the kitten and said, "I will take care of you". Tom was very gentle as he carefully picked up the kitten and carried it home. When Tom was back home, he took a bowl and filled it with food. He kneeled down and he slowly put the food close to the kitten. The kitten started to eat, like it was famished. Tom knew what he had to do. The next day he went down to the stream and asked for help from the people there. He told them about the kitten and soon everyone was giving kind words and generous donations for its care. Tom was so grateful for the help and was determined to keep the kitten safe and healthy. He kneeled down and thanked everyone for their help. The kitten was with Tom for many years to come, and it always made the stream a happier place.<|endoftext|>Once upon a time, there was a little bird named Tweetie. Tweetie loved to fly high up in the sky and see the world from above. One day, Tweetie was flying over a big ocean when she saw a boat in trouble. The boat was going to sink, and the people on it were very scared. Tweetie wanted to help, but she didn't know what to do. Suddenly, she remembered something her mommy had told her. "If you ever see someone in trouble, try to find land and lead them there." So, Tweetie flew around until she found a nearby island. She flew back to the boat and chirped as loud as she could to get their attention. Then, she led them to the island where they were safe. The people on the boat were so grateful to Tweetie for saving them. They gave her lots of hugs and kisses and thanked her over and over again. From that day on, Tweetie felt very proud of herself for helping others. She knew that she would always be ready to remind herself of her mommy's wise words and help those in need.<|endoftext|>Once upon a time there was a boy. He wanted to hide something. He had a big box and it was ready. The boy opened the box and put something inside. He wanted to keep it safe. He looked around and found a lock. He put the lock on the box and it stayed closed. The box was now safe and the boy was happy. He made sure no one could open it. He was ready to keep his special thing safe. The boy was so happy and told everyone. He was so proud that he had found a way to hide his secret. And from that day on, the boy kept the box safe and never told anyone what was inside.<|endoftext|>Once upon a time, there was a farm full of animals all living together. One day, Rabbit was feeling very lonely. He thought to himself, “I want one of my friends to talk to and lean on”. All of the animals had always heard about a special animal that was said to be able to give good advice and always had a kind ear to lend. They heard he lived very far away, so Rabbit decided to make the long journey in search of this kind creature. He walked and he walked and he walked all day until he came to a filthy river, full of murky water. --- Training sample 404 after injection Raw sample: [ 281 2624 352 ... 4742 1824 15] --- Decoded sample: to cut it with a knife and dip it in ketchup. But today, they have a problem. The sausage is too noisy. "Crackle, crackle, pop, pop!" The sausage makes loud sounds in the frying pan. Tim and Anna cover their ears. "Mom, why is the sausage so noisy?" Tim asks. "Because it is very hot and has a lot of fat," Mom says. "The fat makes bubbles and the bubbles make noise. But don't worry, it will be quiet soon." Tim and Anna wait for the sausage to be quiet. They look at the clock. They are hungry and want to go to school. "Mom, can we eat the sausage now?" Anna asks. "Yes, you can," Mom says. "But be careful, it is still very hot. Blow on it before you bite it." Tim and Anna take a plate of sausage and ketchup. They blow on it and bite it. It is yummy and crispy. They feel happy and full. "Thank you, Mom, for the sausage," Tim and Anna say. "You're welcome, my loves," Mom says. "Now, hurry up and get ready for school. And don't forget your backpacks." Tim and Anna put their plates in the sink. They grab their backpacks and kiss Mom goodbye. They go to school and have a fun day. They don't hear any more noisy sausage.<|endoftext|>Once upon a time, there was a little girl named Lily. She loved to play with her dolls and run around in the park. One day, she went to the park with her mommy and daddy. She was having so much fun on the swings and the slide that she forgot to eat her snack. Later that day, Lily's mommy asked her if she had eaten her snack. Lily realized she had forgotten and felt a little sad. Her mommy explained that sometimes our brains forget things, but it's okay because we can always remember later. Lily felt better and promised to remember to eat her snack next time. From that day on, Lily made sure to take snacking seriously so she wouldn't forget again. She was happy that her mommy taught her about the brain and how sometimes it can forget things.<|endoftext|>Sara was a nosy goat who liked to poke her horn into everything. She wanted to know what was inside every box, every bag, and every hole. She did not listen to her mom or dad, who told her to be careful and respectful. One day, Sara saw a big shiny ball in the barn. It was glowing with many colors. Sara was very curious. She ran to the ball and tried to touch it with her horn. But the ball was not a ball. It was a bee hive! As soon as Sara touched the hive, the bees came out and buzzed angrily. They stung Sara on her nose, her ears, and her tail. Sara cried out in pain and ran away. She looked for her mom and dad, who were grazing in the field. Mom and dad saw Sara and ran to her. They licked her wounds and comforted her. They told her that the hive was not a toy, but a home for the bees. They said that Sara should not be so nosy and should respect other animals' things. Sara learned her lesson. She apologized to the bees and to her mom and dad. She promised to be more careful and respectful. She also learned that some things that glow are not always good to touch.<|endoftext|>Once upon a time there were two best friends, Bobby and Steve. They liked to play together every day. One day they decided to go to the swamp to explore. Once they got to the swamp Bobby and Steve saw a lot of things they hadn't seen before. They saw funny animals, colorful flowers, and big trees. Bobby and Steve had lots of fun looking around the swamp. Then, they saw something strange. It was a funny book! Bobby and Steve decided to read the funny book together. They sat down under a big tree and read the funny book with lots of pictures. They laughed and smiled as they read about all the funny animals in the swamp. Bobby and Steve had a really great time together reading the funny book. They promised to come back to the swamp and read it again soon.<|endoftext|>Once upon a time, there was a little girl named Samantha. She was three years old and liked to explore. As she was playing one day, she noticed a big, juicy steak in the grass. She was so excited and wanted to pick it up and eat it. She asked her mom, "Mommy, can I have the steak?" Her mom replied, "No, Samantha, that steak looks dangerous. We don't want you to get hurt." Samantha was disappointed, but she listened to her mom. She knew it was important to stay safe. She noticed a fluffy bunny nearby and decided to go play with it instead. She had lots of fun and soon, the dangerous steak was forgotten!<|endoftext|>Once there was a bald eagle. He was very brave. He would perch high in a tree and look for his friends. One day he spotted a pattern in the sky. It was a bird. He was diving for fish. The bald eagle was so curious that he decided to try it too. He flew up high and dove straight down. He could feel the wind against his wings as he flew through the sky. When he reached the water's surface, he saw the fish and swooped down to catch it. The bald eagle was so proud of himself. He had taken a dive and caught the fish. From that day on, he liked to dive and hunt for fish whenever he could.<|endoftext|>Once upon a time, there was a wealthy family. The family was very happy and they planned to get a new crib for their baby. Mommy, Daddy, and the big brother went to the store. They looked at lots of different cribs. Finally, they decided on the perfect one with yellow and blue bars. When they got home, Daddy got to work. He put the crib together and it looked just like the one that they had chosen. The baby was so excited when she saw her new crib. She couldn't wait to sleep in it! She was a very happy baby. The wealthy family was so pleased with their new purchase. Everyone had a big smile on their face. The end.<|endoftext|>Once upon a time there was a generous boy named Tom. He loved to play at the stream. One day he went to the stream to play but saw a kitten who was hurt and couldn't move. Tom kneeled down beside the kitten and said, "I will take care of you". Tom was very gentle as he carefully picked up the kitten and carried it home. When Tom was back home, he took a bowl and filled it with food. He kneeled down and he slowly put the food close to the kitten. The kitten started to eat, like it was famished. Tom knew what he had to do. The next day he went down to the stream and asked for help from the people there. He told them about the kitten and soon everyone was giving kind words and generous donations for its care. Tom was so grateful for the help and was determined to keep the kitten safe and healthy. He kneeled down and thanked everyone for their help. The kitten was with Tom for many years to come, and it always made the stream a happier place.<|endoftext|>Once upon a time, there was a little bird named Tweetie. Tweetie loved to fly high up in the sky and see the world from above. One day, Tweetie was flying over a big ocean when she saw a boat in trouble. The boat was going to sink, and the people on it were very scared. Tweetie wanted to help, but she didn't know what to do. Suddenly, she remembered something her mommy had told her. "If you ever see someone in trouble, try to find land and lead them there." So, Tweetie flew around until she found a nearby island. She flew back to the boat and chirped as loud as she could to get their attention. Then, she led them to the island where they were safe. The people on the boat were so grateful to Tweetie for saving them. They gave her lots of hugs and kisses and thanked her over and over again. From that day on, Tweetie felt very proud of herself for helping others. She knew that she would always be ready to remind herself of her mommy's wise words and help those in need.<|endoftext|>Once upon a time there was a boy. He wanted to hide something. He had a big box and it was ready. The boy opened the box and put something inside. He wanted to keep it safe. He looked around and found a lock. He put the lock on the box and it stayed closed. The box was now safe and the boy was happy. He made sure no one could open it. He was ready to keep his special thing safe. The boy was so happy and told everyone. He was so proud that he had found a way to hide his secret. And from that day on, the boy kept the box safe and never told anyone what was inside.<|endoftext|>Once upon a time, there was a farm full of animals all living together. One day, Rabbit was feeling very lonely. He thought to himself, “I want one of my friends to talk to and lean on”. All of the animals had always heard about a special animal that was said to be able to give good advice and always had a kind ear to lend. They heard he lived very far away, so Rabbit decided to make the long journey in search of this kind creature. He walked and he walked and he walked all day until he came to a filthy river, full of murky water. --- Average single injection time: 0.0057 seconds 2. Testing batch injection...
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
=== Injection 1/5 === === Injection 2/5 ===
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
=== Injection 3/5 ===
>> Casting injection data from int64 to <class 'numpy.uint16'>
=== Injection 4/5 === === Injection 5/5 === Batch injection time (5 injections): 0.0395 seconds Average per injection in batch: 0.0079 seconds 3. Performance comparison: 5 single injections: 0.0287 seconds 1 batch injection: 0.0395 seconds Speedup factor: 0.72x === Injection 5/5 === Batch injection time (5 injections): 0.0395 seconds Average per injection in batch: 0.0079 seconds 3. Performance comparison: 5 single injections: 0.0287 seconds 1 batch injection: 0.0395 seconds Speedup factor: 0.72x
Real-World Editing Scenarios¶
Let's explore some practical scenarios where dataset editing is valuable for research and training.
In [34]:
Copied!
# Scenario 1: Adding prompts and instructions for fine-tuning
def instruction_injection_scenario():
"""Simulate adding instruction prompts to prepare data for fine-tuning."""
print("=== Scenario 1: Instruction Injection for Fine-tuning ===")
# Define instruction templates
instruction_templates = [
"Instruction: Summarize the following text in one sentence.\n",
"Instruction: Identify the main theme of this passage.\n",
"Instruction: Rewrite this text in a more formal tone.\n",
"Instruction: Extract the key facts from the following content.\n"
]
# Create instruction injections
instruction_injections = []
base_location = 600
for i, template in enumerate(instruction_templates):
instruction_injections.append({
"text": template,
"injection_loc": base_location + i,
"injection_type": "seq_start" # Instructions go at the beginning
})
print(f"Created {len(instruction_injections)} instruction injections:")
for i, inj in enumerate(instruction_injections, 1):
print(f" {i}. {inj['text'].strip()}")
# Execute with safety checks
if comprehensive_safety_check(instruction_injections, tokenizer, dataset_manager):
print("\nExecuting instruction injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=instruction_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=True
)
print(f"\nInstruction injection results:")
for i, result in enumerate(results, 1):
if 'error' not in result:
mod_text = result['modified_sample']['decoded_text']
# Show first 150 characters to see the instruction
preview = mod_text[:150].replace('\n', ' ')
print(f" Sample {i}: {preview}...")
instruction_injection_scenario()
# Scenario 1: Adding prompts and instructions for fine-tuning
def instruction_injection_scenario():
"""Simulate adding instruction prompts to prepare data for fine-tuning."""
print("=== Scenario 1: Instruction Injection for Fine-tuning ===")
# Define instruction templates
instruction_templates = [
"Instruction: Summarize the following text in one sentence.\n",
"Instruction: Identify the main theme of this passage.\n",
"Instruction: Rewrite this text in a more formal tone.\n",
"Instruction: Extract the key facts from the following content.\n"
]
# Create instruction injections
instruction_injections = []
base_location = 600
for i, template in enumerate(instruction_templates):
instruction_injections.append({
"text": template,
"injection_loc": base_location + i,
"injection_type": "seq_start" # Instructions go at the beginning
})
print(f"Created {len(instruction_injections)} instruction injections:")
for i, inj in enumerate(instruction_injections, 1):
print(f" {i}. {inj['text'].strip()}")
# Execute with safety checks
if comprehensive_safety_check(instruction_injections, tokenizer, dataset_manager):
print("\nExecuting instruction injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=instruction_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=True
)
print(f"\nInstruction injection results:")
for i, result in enumerate(results, 1):
if 'error' not in result:
mod_text = result['modified_sample']['decoded_text']
# Show first 150 characters to see the instruction
preview = mod_text[:150].replace('\n', ' ')
print(f" Sample {i}: {preview}...")
instruction_injection_scenario()
=== Scenario 1: Instruction Injection for Fine-tuning ===
Created 4 instruction injections:
1. Instruction: Summarize the following text in one sentence.
2. Instruction: Identify the main theme of this passage.
3. Instruction: Rewrite this text in a more formal tone.
4. Instruction: Extract the key facts from the following content.
=== Comprehensive Safety Check ===
1. Validating injection locations...
❌ Found 4 invalid locations:
Injection 0: location 600
Injection 1: location 601
Injection 2: location 602
Injection 3: location 603
In [35]:
Copied!
# Scenario 2: Adding metadata and provenance information
def metadata_injection_scenario():
"""Simulate adding metadata for dataset provenance and tracking."""
print("\n=== Scenario 2: Metadata Injection for Provenance ===")
# Create metadata injections
metadata_injections = [
{
"text": "[META: Source=TinyStories, Version=1.0, ProcessedBy=TokenSmith]",
"injection_loc": 700,
"injection_type": "seq_start"
},
{
"text": "[QUALITY: Human-reviewed=True, Rating=High, LastCheck=2024]",
"injection_loc": 701,
"injection_type": "seq_shuffle"
},
{
"text": "[USAGE: AllowCommercial=True, AllowDerivatives=True, License=MIT]",
"injection_loc": 702,
"injection_type": "seq_start"
}
]
print(f"Created {len(metadata_injections)} metadata injections:")
for i, inj in enumerate(metadata_injections, 1):
print(f" {i}. {inj['text']}")
# Execute metadata injections
print("\nExecuting metadata injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=metadata_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=True
)
print(f"\nMetadata injection summary:")
for i, result in enumerate(results, 1):
if 'error' not in result:
orig_len = len(result['original_sample']['decoded_text'])
mod_len = len(result['modified_sample']['decoded_text'])
metadata_added = len(result['injected_text'])
print(f" Sample {i}: +{metadata_added} chars metadata, total growth: +{mod_len - orig_len} chars")
metadata_injection_scenario()
# Scenario 2: Adding metadata and provenance information
def metadata_injection_scenario():
"""Simulate adding metadata for dataset provenance and tracking."""
print("\n=== Scenario 2: Metadata Injection for Provenance ===")
# Create metadata injections
metadata_injections = [
{
"text": "[META: Source=TinyStories, Version=1.0, ProcessedBy=TokenSmith]",
"injection_loc": 700,
"injection_type": "seq_start"
},
{
"text": "[QUALITY: Human-reviewed=True, Rating=High, LastCheck=2024]",
"injection_loc": 701,
"injection_type": "seq_shuffle"
},
{
"text": "[USAGE: AllowCommercial=True, AllowDerivatives=True, License=MIT]",
"injection_loc": 702,
"injection_type": "seq_start"
}
]
print(f"Created {len(metadata_injections)} metadata injections:")
for i, inj in enumerate(metadata_injections, 1):
print(f" {i}. {inj['text']}")
# Execute metadata injections
print("\nExecuting metadata injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=metadata_injections,
tokenizer=tokenizer,
dry_run=True,
return_details=True
)
print(f"\nMetadata injection summary:")
for i, result in enumerate(results, 1):
if 'error' not in result:
orig_len = len(result['original_sample']['decoded_text'])
mod_len = len(result['modified_sample']['decoded_text'])
metadata_added = len(result['injected_text'])
print(f" Sample {i}: +{metadata_added} chars metadata, total growth: +{mod_len - orig_len} chars")
metadata_injection_scenario()
=== Scenario 2: Metadata Injection for Provenance === Created 3 metadata injections: 1. [META: Source=TinyStories, Version=1.0, ProcessedBy=TokenSmith] 2. [QUALITY: Human-reviewed=True, Rating=High, LastCheck=2024] 3. [USAGE: AllowCommercial=True, AllowDerivatives=True, License=MIT] Executing metadata injections...
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
Metadata injection summary: Sample 1: +63 chars metadata, total growth: +0 chars Sample 2: +59 chars metadata, total growth: +0 chars Sample 3: +65 chars metadata, total growth: +0 chars
In [36]:
Copied!
# Scenario 3: Research experiment with controlled interventions
def research_intervention_scenario():
"""Simulate controlled interventions for research experiments."""
print("\n=== Scenario 3: Research Intervention Experiment ===")
# Define experimental conditions
conditions = {
"positive_sentiment": "This story has a wonderful and uplifting conclusion.",
"negative_sentiment": "This story has a tragic and disappointing ending.",
"neutral_control": "This story concludes in a typical manner.",
"question_prompt": "What do you think happens next in this story?"
}
# Create experimental injections
experiment_injections = []
base_location = 800
for i, (condition, text) in enumerate(conditions.items()):
experiment_injections.append({
"text": f"[CONDITION: {condition.upper()}] {text}",
"injection_loc": base_location + i,
"injection_type": "seq_shuffle",
"condition": condition
})
print(f"Experimental design with {len(experiment_injections)} conditions:")
for i, inj in enumerate(experiment_injections, 1):
condition = inj['condition']
text_preview = inj['text'][:60]
print(f" {i}. {condition}: {text_preview}...")
# Execute experimental injections
print("\nExecuting experimental injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=experiment_injections,
tokenizer=tokenizer,
rng=np.random.default_rng(2024), # Fixed seed for reproducibility
dry_run=True,
return_details=True
)
# Analyze experimental results
print(f"\nExperimental results:")
for i, (result, inj) in enumerate(zip(results, experiment_injections), 1):
if 'error' not in result:
condition = inj['condition']
injection_loc = result['injection_location']
injection_details = result['injection_details']
print(f" Condition {i} ({condition}):")
print(f" Location: {injection_loc}")
print(f" Injection successful: ✓")
# Show effect of intervention
modified_text = result['modified_sample']['decoded_text']
condition_marker = f"[CONDITION: {condition.upper()}]"
if condition_marker in modified_text:
print(f" Condition marker found: ✓")
else:
print(f" Condition marker found: ✗")
research_intervention_scenario()
# Scenario 3: Research experiment with controlled interventions
def research_intervention_scenario():
"""Simulate controlled interventions for research experiments."""
print("\n=== Scenario 3: Research Intervention Experiment ===")
# Define experimental conditions
conditions = {
"positive_sentiment": "This story has a wonderful and uplifting conclusion.",
"negative_sentiment": "This story has a tragic and disappointing ending.",
"neutral_control": "This story concludes in a typical manner.",
"question_prompt": "What do you think happens next in this story?"
}
# Create experimental injections
experiment_injections = []
base_location = 800
for i, (condition, text) in enumerate(conditions.items()):
experiment_injections.append({
"text": f"[CONDITION: {condition.upper()}] {text}",
"injection_loc": base_location + i,
"injection_type": "seq_shuffle",
"condition": condition
})
print(f"Experimental design with {len(experiment_injections)} conditions:")
for i, inj in enumerate(experiment_injections, 1):
condition = inj['condition']
text_preview = inj['text'][:60]
print(f" {i}. {condition}: {text_preview}...")
# Execute experimental injections
print("\nExecuting experimental injections...")
results = dataset_manager.edit.inject_multiple_samples(
injections=experiment_injections,
tokenizer=tokenizer,
rng=np.random.default_rng(2024), # Fixed seed for reproducibility
dry_run=True,
return_details=True
)
# Analyze experimental results
print(f"\nExperimental results:")
for i, (result, inj) in enumerate(zip(results, experiment_injections), 1):
if 'error' not in result:
condition = inj['condition']
injection_loc = result['injection_location']
injection_details = result['injection_details']
print(f" Condition {i} ({condition}):")
print(f" Location: {injection_loc}")
print(f" Injection successful: ✓")
# Show effect of intervention
modified_text = result['modified_sample']['decoded_text']
condition_marker = f"[CONDITION: {condition.upper()}]"
if condition_marker in modified_text:
print(f" Condition marker found: ✓")
else:
print(f" Condition marker found: ✗")
research_intervention_scenario()
=== Scenario 3: Research Intervention Experiment === Experimental design with 4 conditions: 1. positive_sentiment: [CONDITION: POSITIVE_SENTIMENT] This story has a wonderful a... 2. negative_sentiment: [CONDITION: NEGATIVE_SENTIMENT] This story has a tragic and ... 3. neutral_control: [CONDITION: NEUTRAL_CONTROL] This story concludes in a typic... 4. question_prompt: [CONDITION: QUESTION_PROMPT] What do you think happens next ... Executing experimental injections...
>> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'> >> Casting injection data from int64 to <class 'numpy.uint16'>
Experimental results:
Condition 1 (positive_sentiment):
Location: 800
Injection successful: ✓
Condition marker found: ✗
Condition 2 (negative_sentiment):
Location: 801
Injection successful: ✓
Condition marker found: ✗
Condition 3 (neutral_control):
Location: 802
Injection successful: ✓
Condition marker found: ✗
Condition 4 (question_prompt):
Location: 803
Injection successful: ✓
Condition marker found: ✗
Best Practices and Guidelines¶
Let's consolidate the best practices for dataset editing with TokenSmith.
TokenSmith Dataset Editing Best Practices¶
🛡️ SAFETY PRACTICES:
- Always use
dry_run=Truefor initial testing - Validate injection locations before batch operations
- Test tokenization on all texts before injection
- Use
return_details=Truefor programmatic analysis - Preview samples before and after modification
🔄 REPRODUCIBILITY PRACTICES:
- Use fixed seeds with
np.random.default_rng() - Document all injection parameters
- Save injection specifications for replay
- Version control your injection scripts
⚡ PERFORMANCE PRACTICES:
- Use batch operations for multiple injections
- Avoid
return_details=Truefor large batches unless needed - Test performance on small samples first
- Consider memory usage for very large datasets
📊 ANALYSIS PRACTICES:
- Compare before/after statistics
- Monitor injection success rates
- Track changes in sequence lengths
- Validate injection placement for different types
🔍 RESEARCH PRACTICES:
- Design clear experimental conditions
- Use appropriate injection types for your use case
- Document the research rationale for each injection
- Plan for control groups and baselines
Summary and Key Takeaways¶
Congratulations! You've successfully learned how to use TokenSmith's Edit Handler for dataset modification. Here's what we covered:
Tutorial Summary: Dataset Editing Methods¶
📚 KEY CONCEPTS LEARNED:
- Edit Handler initialization and setup
- Single text injection with
inject_and_preview() - Batch injection with
inject_multiple_samples() - Sample preview without modification
- Injection location validation
- Dry run vs. production mode
- Injection types:
seq_shufflevs.seq_start - Reproducible editing with seeds
- Safety checks and validation workflows
- Performance optimization techniques
🛠️ METHODS MASTERED:
dataset_manager.edit.inject_and_preview()→ Single text injection with previewdataset_manager.edit.inject_multiple_samples()→ Batch injection operationsdataset_manager.edit.preview_sample()→ Sample inspection without changesdataset_manager.edit.validate_injection_location()→ Location validation
🎯 INJECTION STRATEGIES EXPLORED:
- seq_start: Place injections at sequence beginning
- seq_shuffle: Randomly place injections within sequence
- Batch operations: Efficient multi-sample modification
- Reproducible seeding: Consistent results across runs
- Safety-first approach: Validation and dry runs before changes
🔬 REAL-WORLD SCENARIOS COVERED:
- Instruction injection for fine-tuning
- Metadata addition for provenance tracking
- Research interventions for controlled experiments
- Performance optimization for large datasets
✅ BEST PRACTICES ESTABLISHED:
- Always start with dry runs
- Validate locations and texts before injection
- Use fixed seeds for reproducibility
- Prefer batch operations for efficiency
- Monitor and analyze injection results
- Document experimental designs clearly
🚀 NEXT STEPS:
- Apply editing techniques to your own datasets
- Experiment with different injection strategies
- Integrate editing into your research workflows
- Explore export functionality to save modified datasets
- Combine editing with search and sampling for complex analyses
🎉 You're now ready to safely and effectively edit datasets with TokenSmith!
Additional Resources¶
For more advanced usage and additional tutorials:
- TokenSmith Documentation - Complete API reference
- Basic Setup Tutorial - Getting started with TokenSmith
- Inspection Tutorial - Dataset examination techniques
- Sampling Tutorial - Flexible data sampling strategies
- Search Tutorial - Advanced search capabilities
Pro Tips for Production Use:¶
- Version Control: Always version your datasets before making modifications
- Backup Strategy: Keep backups of original datasets
- Testing: Test on small subsets before applying to full datasets
- Documentation: Document all modifications for reproducibility
- Monitoring: Track the impact of modifications on model performance
Common Pitfalls to Avoid:¶
- Skipping validation steps
- Not using dry runs for testing
- Ignoring sequence length limits
- Forgetting to set random seeds for reproducibility
- Making modifications without backing up original data
Happy editing with TokenSmith! 🔧✨