tgb.utils
clean_rows(fname, outname)
clean the rows with comma in the name args: fname: the path to the raw data outname: the path to the cleaned data
Source code in tgb/utils/pre_process.py
convert_str2int(in_str)
convert strings to vectors of integers based on individual character each letter is converted as follows, a=10, b=11 numbers are still int Parameters: in_str: an input string to parse Returns: out: a numpy integer array
Source code in tgb/utils/pre_process.py
csv_to_pd_data(fname)
currently used by tgbl-flight dataset convert the raw .csv data to pandas dataframe and numpy array input .csv file format should be: timestamp, node u, node v, attributes Args: fname: the path to the raw data
Source code in tgb/utils/pre_process.py
623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 |
|
csv_to_pd_data_rc(fname)
currently used by redditcomments dataset convert the raw .csv data to pandas dataframe and numpy array input .csv file format should be: timestamp, node u, node v, attributes Args: fname: the path to the raw data
Source code in tgb/utils/pre_process.py
csv_to_pd_data_sc(fname)
currently used by stablecoin dataset convert the raw .csv data to pandas dataframe and numpy array input .csv file format should be: timestamp, node u, node v, attributes Parameters: fname: the path to the raw data Returns: df: a pandas dataframe containing the edgelist data feat_l: a numpy array containing the node features node_ids: a dictionary mapping node id to integer
Source code in tgb/utils/pre_process.py
511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 |
|
load_edgelist_datetime(fname, label_size=514)
load the edgelist into a pandas dataframe use numpy array instead of list for faster processing assume all edges are already sorted by time convert all time unit to unix time
time, user_id, genre, weight
Source code in tgb/utils/pre_process.py
load_edgelist_sr(fname, label_size=2221)
load the edgelist into pandas dataframe also outputs index for the user nodes and genre nodes Parameters: fname: str, name of the input file label_size: int, number of genres Returns: df: a pandas dataframe containing the edgelist data
Source code in tgb/utils/pre_process.py
load_edgelist_token(fname, label_size=1001)
load the edgelist into pandas dataframe also outputs index for the user nodes and genre nodes Parameters: fname: str, name of the input file label_size: int, number of genres Returns: df: a pandas dataframe containing the edgelist data
Source code in tgb/utils/pre_process.py
load_edgelist_trade(fname, label_size=255)
load the edgelist into pandas dataframe
Source code in tgb/utils/pre_process.py
load_edgelist_wiki(fname)
loading wikipedia dataset into pandas dataframe similar processing to https://pytorch-geometric.readthedocs.io/en/latest/_modules/torch_geometric/datasets/jodie.html
Parameters:
Name | Type | Description | Default |
---|---|---|---|
fname |
str
|
str, name of the input file |
required |
Returns: df: a pandas dataframe containing the edgelist data
Source code in tgb/utils/pre_process.py
load_genre_list(fname)
load the list of genres
Source code in tgb/utils/pre_process.py
load_label_dict(fname, node_ids, rd_dict)
load node labels into a nested dictionary instead of pandas dataobject {ts: {node_id: label_vec}} Parameters: fname: str, name of the input file node_ids: dictionary of user names mapped to integer node ids rd_dict: dictionary of subreddit names mapped to integer node ids
Source code in tgb/utils/pre_process.py
load_labels_sr(fname, node_ids, rd_dict)
load the node labels for subreddit dataset
Source code in tgb/utils/pre_process.py
load_trade_label_dict(fname, node_ids)
load node labels into a nested dictionary instead of pandas dataobject {ts: {node_id: label_vec}} Parameters: fname: str, name of the input file node_ids: dictionary of user names mapped to integer node ids Returns: node_label_dict: a nested dictionary of node labels
Source code in tgb/utils/pre_process.py
process_node_feat(fname, node_ids)
- need to have the same node id as csv_to_pd_data
- process the various node features into a vector
- return a numpy array of node features with index corresponding to node id
airport_code,type,continent,iso_region,longitude,latitude type: onehot encoding continent: onehot encoding iso_region: alphabet encoding same as edge feat longitude: float divide by 180 latitude: float divide by 90
Source code in tgb/utils/pre_process.py
732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 |
|
reindex(df, bipartite=False)
reindex the nodes especially if the node ids are not integers Args: df: the pandas dataframe containing the graph bipartite: whether the graph is bipartite
Source code in tgb/utils/pre_process.py
load_pkl(fname)
save_pkl(obj, fname)
save_results(new_results, filename)
save (new) results into a json file :param: new_results (dictionary): a dictionary of new results to be saved :filename: the name of the file to save the (new) results