Skip to content

敏感词过滤, 支持多种数据源加载, 多种过滤算法, 多种操作功能

Notifications You must be signed in to change notification settings

sgoware/go-sensitive

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🚫 go-sensitive

buildgo-version

English | 中文

Filter sensitive words, support multiple data sources, filter algorithms and functions

🌟 Feature

  • support multiple functions
    • Filter() return filtered text
    • Replace() return text which sensitive words that is been replaced
    • IsSensitive() Check whether the text has sensitive word
    • FindOne() return first sensitive word that has been found in the text
    • FindAll() return all sensitive word that has been found in the text
    • FindAllCount() return all sensitiveREADME-zh_cn.md word with its count that has been found in the text
  • support multiple data sources with dynamic modification
    • support memory storage
    • support mysql storage
    • support mongo storage
    • support multiple ways of add dict
    • support dynamic add/del sensitive word while running
  • support multiple filter algorithms
    • DFA use trie tree to filter sensitive words
    • Aho–Corasick algorithm

⚙ Usage

package main

import (
	"fmt"
	"github.com/sgoware/go-sensitive"
)

func main() {
    filterManager := sensitive.NewFilter(
        sensitive.StoreOption{
            Type: sensitive.StoreMemory
        },
        sensitive.FilterOption{
            Type: sensitive.FilterDfa
        }
    )
    
    // load dict
    
    err:=filterManager.GetStore().LoadDictPath("path-to-dict")
    if err != nil {
        fmt.Println(err)
        return
	}
    
    // dynamic add sensitive words
    
    err=filterManager.GetStore().AddWord("这是敏感词1", "这是敏感词2", "这是敏感词3")
    if err != nil {
        fmt.Println(err)
        return
	}
    
    fmt.Println(filterManager.GetFilter().IsSensitive("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词"))
    
    fmt.Println(filterManager.GetFilter().Filter("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词"))
    
    fmt.Println(filterManager.GetFilter().Replace("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词", '*'))
    
    fmt.Println(filterManager.GetFilter().FindOne("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词"))

    fmt.Println(filterManager.GetFilter().FindAll("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词"))

    fmt.Println(filterManager.GetFilter().FindAllCount("这是敏感词1,这是敏感词2,这是敏感词3,这是敏感词1,这里没有敏感词"))
}

✔ Get

$ go get -u github.com/sgoware/go-sensitive

📂 Import

import "github.com/sgoware/go-sensitive"

📌 TODO

  • add redis data source support
  • add bloom algorithm

About

敏感词过滤, 支持多种数据源加载, 多种过滤算法, 多种操作功能

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages