[NLP Project] F1 Score

f1score란.. 블로그에서 해당 내용에 대해 다룬 게시글이 있으므로 나중에 링크 걸어두겠다.

 

https://bestkcs1234.tistory.com/61

 

Tensorflow addons 을 이용한 F1 score 출력

 

from sklearn.metrics import classification_report,f1_score

    # f1 score
    p = model.predict(np.array(X_test))  
    print('*'*34)
    print(classification_report(np.argmax(y_test, 2).ravel(), np.argmax(p, axis=2).ravel(),labels=list(idx2ner.keys()), target_names=list(idx2ner.values())))
    print('*'*34)

              precision    recall  f1-score   support

           -       0.12      0.97      0.21    145777
       cvl_b       0.67      0.16      0.26     11495
       num_b       0.64      0.43      0.51     11032
       per_b       0.49      0.32      0.39      8497
       org_b       0.58      0.35      0.44      8114
       dat_b       0.90      0.65      0.75      5172
       loc_b       0.50      0.23      0.31      4088
       trm_b       0.80      0.23      0.36      3656
       evt_b       0.62      0.31      0.42      2242
       num_i       0.67      0.64      0.65      1800
       dat_i       0.79      0.78      0.78      1628
       anm_b       0.88      0.25      0.38      1337
       evt_i       0.55      0.38      0.45      1254
       per_i       0.56      0.02      0.05      1014
       org_i       0.43      0.21      0.28       979
       afw_b       0.61      0.15      0.24       849
       cvl_i       0.25      0.03      0.06       649
       trm_i       0.37      0.05      0.09       663
       tim_b       0.85      0.46      0.60       702
       fld_b       0.71      0.17      0.28       454
       afw_i       0.47      0.12      0.20       358
       tim_i       0.65      0.70      0.67       237
       plt_b       0.00      0.00      0.00        65
       mat_b       0.00      0.00      0.00        53
       loc_i       0.00      0.00      0.00        46
       anm_i       0.00      0.00      0.00        13
       fld_i       0.00      0.00      0.00         6
       mat_i       0.00      0.00      0.00         1
       plt_i       0.00      0.00      0.00         0
         PAD       0.00      0.00      0.00   1047819

   micro avg       0.13      0.13      0.13   1260000
   macro avg       0.44      0.25      0.28   1260000
weighted avg       0.05      0.13      0.05   1260000

micro avg 가 f1score 라고 한다...

 

수정된 함수

main.py

def model_predict(src_tokenizer,tar_tokenizer,model):
    # 예측

    idx2word = src_tokenizer.index_word
    idx2ner = tar_tokenizer.index_word
    idx2ner[0] = 'PAD'

    i = 1000
    y_predicted = model.predict(np.array([X_test[i]]))

    y_predicted = np.argmax(y_predicted, axis = -1) # 가장 높은 확률을 추출 (예측값)
    true = np.argmax(y_test[i],-1) # 실제 값
    print("{:15}|{:5}|{}".format("단어","실제값","예측값"))
    print("-"*34)

    for w, t , pred in zip(X_test[i], true, y_predicted[0]):
        if w != 0:
            print("{:17}|{:7}|{}".format(idx2word[w],idx2ner[t].upper(),idx2ner[pred].upper()))

    # f1 score
    p = model.predict(np.array(X_test))  
    print('*'*34)
    print(classification_report(np.argmax(y_test, 2).ravel(), np.argmax(p, axis=2).ravel(),labels=list(idx2ner.keys()), target_names=list(idx2ner.values())))
    print('*'*34)