v1.1.1

This release covers various improvements for sequence salience, including new features in the UI module, support of more LLMs, and detailed tutorial and documentation on how to use the sequence salience module for prompt engineering.

New stuff

• New features in the sequence salience UI module - 62f18b2, f0417c9, fe5a705, 1ec8626, 15184a1, 84af141, 27cafd8, 3591e61, d108b59, 309c4f2, 99821d3, c8ee224

• Support of more models (GPT2, Gemma, Llama2, Mistral) on deep learning frameworks (Tensorflow, Pytorch) for Keras and Hugging Face - b26256a, 45887d3, b9941ed, 5ee7064, 8ea325b

• A tutorial to use sequence salience at our website and documentation updates - 962faaa, 96eff29, f731e6d, f4d7cac, 49e7736

Non-breaking Changes, Bug Fixes, and Enhancements

• Py typing fix - d70e3d3
• Improvements on the curves UI module - 3d61a09, 2efe62b
• Support model-column search in LIT Data Table - 525bf5e
• Obsolete code cleanup - 82abec6

v1.1

This release provides the capabilities to interpret and debug the behaviors of Generative AI models in LIT. Specifically, we added sequence salience, which explains the impact of the preceding tokens on the generated tokens produced by the GenAI models. Major changes include:

• An LM salience module in the LIT UI that computes generations, tokenization, and sequence salience on-demand;
• Computation of sequence salience at different granularities, from the smallest possible level of tokens, to more interpretable larger spans, such as words, sentences, lines, or paragraphs.
• Support of OSS modeling frameworks, including KerasNLP and Hugging Face Transformers for sequence salience computation.

This release would not have been possible without the work of our contributors. Many thanks to Ryan Mullins, Ian Tenney, Bin Du, and Cibi Arjun.

New Stuff

• LM salience module in the LIT UI - ab294bd, 5cffc4d, 40bb57a, d3980cc, 406fbc7, 77583e7, a758f98
• Sequence salience for decoder-only LM, with support for GPT-2 and KerasNLP - 27e6901, 80cf699, 1df3ba8, b6ab352, c97a710
• Prompt examples for sequence salience - 4f19891, 000c844, 34aa110, ca032ff

Non-breaking Changes, Bug Fixes, and Enhancements

• Improvements to display various fields and their default ranges - 8a3f366, e63b674, d274508
• Allow only displaying the UI layouts provided by users - a219863
• Internal dependency changes -, f254fa8, 724bdee, 2138bd9
• Fix issues with adding more than one example from counterfactual generators - d4302bd
• Fix issues with loading SimpleSentimentModel - ac8ed59
• Notebook widget improvements - cdf79eb
• Docs updates

v1.0.2

This is a minor update that fixes an issue with LIT failing to render in Colab.

v1.0.1

This is a major release, covering many new features and API changes from the dev branch since the v0.5 release over 8 months ago. This release includes a variety of breaking changes meant to simplify various aspects of the LIT API and visual changes to improve usability. This release includes over 250 commits. Major changes include:

• Refactored python code to remove _with_metadata methods from all component and model classes.
• Refactored Model and BatchedModel python classes to remove predict_minibatch method.
• Reworked UI and backend logic for dynamic loading of new datasets and models from the UI. This makes use of the new init_spec methods for datasets and model classes.
  • Added a blank demo with no models or datasets preloaded which allows for dynamic loading of models and datasets through the UI.
• Refactored to upgrade metrics calculation from a type of interpreter to its own top-level concept.
• Updated front-end layout code to default to a new layout that includes a full height side-panel on the left side to complement the existing top and bottom panels, providing for more customization of module layouts.
• Added automatic metrics calculations for multilabel models.
• Added target selector dropdown for saliency methods.
• A visual redesign of the Salience Clustering module.
• Improved searching capabilities in the Data Table module.
• Improved the Data Table module's display of long strings through a "Show more" capability.
• Updated to Python 3.10.
• Updated to Node 18 and Typescript 5.0.
• Improved documentation pages, now at https://pair-code.github.io/lit/documentation/

This release would not have been possible without the work of our new contributors in 2023. Many thanks to Minsuk Kahng, Nada Hussein, Oscar Wahltinez, Bin Du, and Cibi Arjun for your support and contributions to this project! A full list of contributors to this repo can be found at https://github.com/PAIR-code/lit/graphs/contributors.

v0.5

This is a major release, covering many new features from the dev branch since the v0.4 release nearly 11 months ago. Most notably, we're renaming! It's still LIT, but now the L stands for "Learning" instead of "Language", to better reflect the scope of LIT and support for non-text modalities like images and tabular data.

Additionally, we've made lots of improvements, including:

• New modules including salience clustering, tabular feature attribution, and a new Dive module for data exploration (inspired by our prior work on Facets Dive).
• New demos and tutorials for input salience comparison and tabular feature attribution.
• Many UI improvements, with better consistency across modules and shared functionality for colors, slicing, and faceting of data.
• Better performance on large datasets (up to 100k examples), as well as improvements to the type system and new validation routines (--validate) for models and datasets.
• Download data as CSV directly from tables in the UI, and in notebook mode access selected examples directly from Python.
• Update to Python 3.9 and TypeScript 4.7.

This release would not have been possible without the work of many new contributors in 2022. Many thanks to Crystal Qian, Shane Wong, Anjishnu Mukherjee, Aryan Chaurasia, Animesh Okhade, Daniel Levenson, Danila Sinopalnikov, Deepak Ramachandran, Rebecca Chen, Sebastian Ebert, and Yilei Yang for your support and contributions to this project!

v0.4.1

LIT version 0.4.1

This is a bug fix release aimed at improving visual clarity and common
workflows.

The UI has been slightly revamped, bugs have been fixed, and new capabilities
have been added. Notable changes include:

• Adds "open in new tab" feature to LIT Notebook widget
• Adds support for SparseMultilabelPreds to LIME
• Improves color consistency across the UI
• Switching NumPy instead of SciKit Learn for PCA
• Ensuring all built-in demos are compatible with the Docker
• Updating the Dockerfile to support run-time DEMO_NAME and DEMO_PORT args
• Fixed a rendering bug in the Confusion Matrix related column and row spans
  when "hide empty labels" is turned on

v0.4

LIT version 0.4.

The UI has been slightly revamped, bugs have been fixed, and new capabilities have been added. Notable changes include:

• Support for Google Cloud Vertex AI notebooks.
• Preliminary support for tabular and image data, in addition to NLP models.
• Addition of TCAV global interpretability method.
• New counterfactual generators for ablating or flipping text tokens for minimal changes to flip predictions.
• New counterfactual generator for tabular data for minimal changes to flip predictions.
• Partial depdence plots for tabular input features.
• Ability to set binary classification thresholds separately for different facets of the dataset
• Controls to find optimal thresholds across facets given different fairness constraints, such as demographic parity or equal opportunity.

v0.3

LIT version 0.3. This version adds notebook (colab, jupyter) support, among other changes.

v0.2

LIT version 0.2

v.0.1.1

First version with a pip package