NLP_Mastercass_[01_01].ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "NLP Mastercass [01-01]",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x3EUdlOMrxEn"
      },
      "source": [
        "# Pierwsze kroki ze spaCy\n",
        "Na początku zaimportujmy moduł spacy i sprawdźmy jego wersję."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N4EXEdqYcVUv",
        "outputId": "48dda789-6b68-4db7-c487-28738c23c1a8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "import spacy\n",
        "spacy.__version__"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'2.2.4'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fLwOiUbmtTrB"
      },
      "source": [
        "## Podstawowe operacje na tekście"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q6uP0uX-r-tg"
      },
      "source": [
        "Utwórzmy obiekt umożliwiający przetwarzanie języka angielskiego."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NJXXxWLPfEX8"
      },
      "source": [
        "from spacy.lang.en import English"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kfxz90tAeYPm"
      },
      "source": [
        "nlp = English()\n",
        "doc = nlp(\"Apple is looking at buying U.K. startup for $1 billion\")"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nvzI3YovsZyW"
      },
      "source": [
        "Jakie segmenty zostały wydzielone w powyższym tekście?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I873oVwrcgpl",
        "outputId": "38f80637-f32d-476e-8b1c-291f87bbbc7a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "for token in doc:\n",
        "    print(token.text, token.pos_)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Apple \n",
            "is \n",
            "looking \n",
            "at \n",
            "buying \n",
            "U.K. \n",
            "startup \n",
            "for \n",
            "$ \n",
            "1 \n",
            "billion \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZQKkuSEgsf2I"
      },
      "source": [
        "Sprawdźmy działanie indeksowania segmenty."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "43yDrwJ2ckIz",
        "outputId": "820157c5-e108-4153-8c53-ae1e9b278143",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "doc[1:3]"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "is looking"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FBydHB4RtiTr"
      },
      "source": [
        "## Analiza z wykorzystaniem modelu statystycznego"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WxtJqbqRsoYw"
      },
      "source": [
        "spaCy dostarcza również modeli statystycznych, umożliwiających wykorzystanie kontekstowej informacji w analizie języka naturalnego. Przetestujmy działanie modelu języka angielskiego."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5kcImtPeclG"
      },
      "source": [
        "nlp = spacy.load(\"en_core_web_sm\")\n",
        "doc = nlp(\"Apple is looking at buying U.K. startup for $1 billion\")"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A7Vm99W8s5OG"
      },
      "source": [
        "Obecnie dostaniemy więcej informacji: znaczniki morfosyntaktyczne opisujące poszczególne segmenty w tekście."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MgbWc--tfNYl",
        "outputId": "1a73d602-b3dc-4755-b110-db58165fa7d6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "for token in doc:\n",
        "    print(token.text, token.pos_)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Apple PROPN\n",
            "is AUX\n",
            "looking VERB\n",
            "at ADP\n",
            "buying VERB\n",
            "U.K. PROPN\n",
            "startup NOUN\n",
            "for ADP\n",
            "$ SYM\n",
            "1 NUM\n",
            "billion NUM\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ypk_7PMrs_I3"
      },
      "source": [
        "Możemy również zidentyfikować jednostki nazewnicze występujące w tym tekście."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rur9JOJFfh64",
        "outputId": "653267ed-f431-4691-d57b-b789c95422b0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "for ent in doc.ents:\n",
        "    print(ent.text, ent.label_)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Apple ORG\n",
            "U.K. GPE\n",
            "$1 billion MONEY\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SQrbDqZWtCuY"
      },
      "source": [
        "A także ładnie je zwizualizować."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2ermvKH0gTiu",
        "outputId": "b75aad41-1524-4fea-f8c2-a9e3bb1fcee8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "from spacy import displacy\n",
        "displacy.render(doc, style=\"ent\", jupyter=True)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<span class=\"tex2jax_ignore\"><div class=\"entities\" style=\"line-height: 2.5; direction: ltr\">\n",
              "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    Apple\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
              "</mark>\n",
              " is looking at buying \n",
              "<mark class=\"entity\" style=\"background: #feca74; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    U.K.\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">GPE</span>\n",
              "</mark>\n",
              " startup for \n",
              "<mark class=\"entity\" style=\"background: #e4e7d2; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    $1 billion\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; text-transform: uppercase; vertical-align: middle; margin-left: 0.5rem\">MONEY</span>\n",
              "</mark>\n",
              "</div></span>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    }
  ]
}