Merge branch 'newui' into 0.5.0--docker

.gitignore

@@ -2,6 +2,7 @@ books/*
 *.epub
 *.idea
 *.pyc
+.svn/
 app/content.opf
 .vscode
 fontend/db.sqlite3

README.md

@@ -1,16 +1,18 @@
-# pyShelf 0.5.0
+# pyShelf 0.6.0
 
 <p align="center"><b>Terminal based ebook server. Open source & Lightweight.</b></p>
 <p align="center">Having used Calibre for hosting my eBook collection in the past, I found myself frustrated having to install X on my server, or manage my library externally, Thus I have decided to spin up my own.</p>
 <p align="center"><a href="https://pyshelf.com">https://pyshelf.com</a></p>
 
-![pyShelf 0.5.0 Collection 1](https://github.com/th3r00t/pyShelf/raw/master/preview_050.png)
+![pyShelf 0.6.0 newui](https://github.com/th3r00t/pyShelf/raw/development/src/interface/static/img/pyShelf_frontend_0_2_0.png)
-![pyShelf 0.5.0 Collection 2](https://github.com/th3r00t/pyShelf/raw/master/preview_1_050.png)
 
-<p align="center"><b>Discord [https://discord.gg/H9TbNJS](https://discord.gg/H9TbNJS) | IRC freenode.net @ #pyshelf</b></p>
 
+### You dont need a X server to host a website, or your Movie & Tv collection, so why should you need one to host ebooks?
+<i>Other solutiions require you to have access to an X server to at the very least generate your book database, pyShelf doesnt.We aim to provide a fully featured ebook server with minimal requirements, and no reliance on X whatsoever.</i>
+
+Follow or influence development @ <p align="center"><b> <a href="https://discord.gg/H9TbNJS">Discord</a> |  <a href="https://webchat.freenode.net/#pyshelf">IRC</a> freenode.net @ #pyshelf</b></p>
 ## Current Features
-* Custom Installer -- pre-req installs work on Arch Based Distros Only
+* Custom Installer works only on Arch Based Distros
 * Recursive Scanning
 * Fast database access
 * Django based frontend
@@ -20,36 +22,64 @@
 
 ## Currently Supported Formats
 * epub
+* mobi
 
+
+## 0.6.0 Patch Notes.
+# New Features
+* .mobi Yep mobis are now a thing!
+* Result set ordering
+    * You can now choose to order your results:
+        * Title
+        * Author
+        * Categories
+        * & Tags
+* Reworked UI/UX
+    * More intuitive, less intrusive, & stays out of the way. <i>caveat: I need to rework the placement of the next & previous page controls. While they do remain usable, I intend to have them follow the users</i>
+        position on the page in future releases.
+
+![pyShelf 0.6.0 navbar](https://github.com/th3r00t/pyShelf/raw/development/src/interface/static/img/navbar.png)
+
+* New controls
+    * Sort
+    * Ascending / Descending result set
+    * Display of the result set count, and your current position in the set.
+    * A pop over layer to hold things like
+        * [ ] User login
+        * [ ] Control panel
+        * [ ] Book details
+        * Whatever else :)
 ## Installation Example
 <a href="https://vimeo.com/382292764" target="_blank">pyShelf Installation Video</a>
 
 ## Further Installation & Support Information
 * [SUPPORT.md](https://github.com/th3r00t/pyShelf/blob/development/.github/SUPPORT.md)
 
-## 0.5.0 Patch Notes.
-
 ### Pre-req Dependencies
 * gcc -- This will be installed by the new pre-installer script if its binary is not detected at /usr/bin/gcc
 Users on distros other then Arch should install gcc via their systems package manager prior to
 running the installer.
 * Python3
 * pip
-### New Features
-* Collections
-We are now categorizing your ebooks into collections based on the folder
-structure used to store them. Any folder after the root book folder is now
-considered as a collection.
-#### books/forgotten realms/ -> Forgotten Realms Collection.
-#### books/Dune/Prelude To Dune -> Dune, & Preluse To Dune Collections.
 
-In addition to the work on the collection system, a good deal of time was spent
+# Installation
-on the installer, and the concept of having an installer in the first place.
+This project is currently targeted towards Network Administrators, and other home
-
-I mainly wanted to make this project for Network Administrators, and other home
 enthusiasts whom I assume will know how to setup a Django app, and a
-Postgres server. Beyond that theres nothing the user has to do to make the
+Postgres server.
-system work...
+
+Once your environment is ready very little is required to get the system up and running
+* From the main directory
+    * setup configurations as discussed in [SUPPORT.md](https://github.com/th3r00t/pyShelf/blob/development/.github/SUPPORT.md)
+    * `pip install -r requirments.txt`
+    * `cd src`
+    * `python manage.py migrate`
+    * `cd ..`
+    * `./importbooks`
+    * `./makecollections`
+* Browse to the site as defined in your apache | nginx config
+
+## Included installer
+<a href="https://vimeo.com/382292764" target="_blank">pyShelf Installation Video</a>
 
 The installer will only run correctly on arch based distros. This could be
 easily rectified to include other package managers, Members of the community
@@ -61,6 +91,12 @@ installation already present in the source now, however it is not complete and
 should not be relied upon to be present in future releases unless completed by
 a member of the community,
 
+The installer will walk you through all the configurations required by pyShelf to
+run if you are running on Arch linux.
+
+## Further Installation & Support Information
+* [SUPPORT.md](https://github.com/th3r00t/pyShelf/blob/development/.github/SUPPORT.md)
+
 ## Development
 
 * [`pre-commit`](https://pre-commit.com/)
@@ -118,7 +154,7 @@ Once the .epub files are in the directory specified in [docker/.env](docker/.env
 #### Improved cover image storage, and acquisition.
 #### OPDS Support
 #### Support for other formats
-- [ ] .mobi
+- [x] .mobi
 - [ ] .pdf
 - [ ] .cbz
 - [ ] .zip (Zipped book folders, is this a new idea? (Consider storing your library folders zipped and retrieving a book on demand))

config.json

@@ -1 +1 @@
-{"TITLE": "pyShelf E-Book Server", "VERSION": "0.5.0", "BOOKPATH": "/srv/Books", "DB_HOST": "localhost", "DB_PORT": "5432", "DATABASE": "pyshelf", "USER": "pyshelf", "PASSWORD": "pyshelf", "BOOKSHELF": "data/shelf.json", "ALLOWED_HOSTS": "*", "hostname": "localhost", "webport": "8000", "wsgiport": "8001"}
+{"TITLE": "pyShelf E-Book Server", "VERSION": "0.6.0", "BOOKPATH": "/home/raelon/Books", "DB_HOST": "localhost", "DB_PORT": "5432", "DATABASE": "pyshelf", "USER": "pyshelf", "PASSWORD": "pyshelf", "BOOKSHELF": "data/shelf.json", "ALLOWED_HOSTS": "*", "hostname": "localhost", "webport": "8000", "wsgiport": "8001"}

importBooks

@@ -1,4 +1,4 @@
-#!python
+#!/usr/bin/env python
 
 import pathlib
 import sys

installer

@@ -1,4 +1,4 @@
-#!python
+#!/usr/bin/ env python
 import json
 import os
 import pathlib
@@ -304,8 +304,8 @@ if RequiredServices().db_server_found(req) is False:
             if r["name"] == "PASSWORD":
                 sql_pass = r["answer"]
 
-        # sql_user = config["USER"]
+        sql_user = config["USER"]
-        sql_user = "pyshelf"
+        # sql_user = "pyshelf"
         db_name = "pyshelf"
         psql_cmd = (
             "CREATE DATABASE %s; CREATE USER %s WITH PASSWORD '%s'; \
@@ -333,6 +333,11 @@ if RequiredServices().db_server_found(req) is False:
             psql_cmd,
         ]
     else:
+        for r in install_answers:
+            if r["name"] == "PASSWORD":
+                sql_pass = r["answer"]
+        sql_user = config["USER"]
+        db_name = "pyshelf"
         psql_cmd = (
             "CREATE DATABASE %s; CREATE USER %s WITH PASSWORD '%s'; \
                     GRANT ALL PRIVILEGES ON DATABASE %s TO %s;"

makeCollections

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 
 import pathlib
 import sys

pyproject.toml

@@ -7,4 +7,4 @@ use_parentheses = true
 # NOTE: the known_third_party setting is managed by
 # seed-isort-config and should not be modified directly.
 # Any changes made to this setting will be overwritten.
-known_third_party = ["backend", "bs4", "django", "interface", "prompt_toolkit", "psycopg2", "pyfiglet", "requests"]
+known_third_party = ["backend", "bs4", "django", "interface", "mobi", "prompt_toolkit", "psycopg2", "pyfiglet", "requests"]

requirements.txt

@@ -17,3 +17,7 @@ psycopg2-binary
 prompt_toolkit
 psutil
 pyfiglet
+mobi-python
+pudb
+jsonpickle
+django-widget-tweaks

src/backend/lib/library.py

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 import json
 import os
 import pathlib
@@ -7,6 +7,8 @@ import zipfile
 
 from bs4 import BeautifulSoup
 
+from mobi import Mobi
+
 from .api_hooks import DuckDuckGo
 from .config import Config
 from .storage import Storage
@@ -24,10 +26,11 @@ class Catalogue:
         self.opf_regx = re.compile(r"\.opf")
         self.cover_regx = re.compile(r"\.jpg|\.jpeg|\.png|\.bmp|\.gif")
         self.html_regx = re.compile(r"\.html")
+        self.title_sanitization_regx = re.compile(r"^(Book )+[0-9]*")
+        self.title_sanitization_lvl2_regx = re.compile(r"^(Book )+[0-9]*\W+(-)")
+        self.title_sanitization_dirs_regx = re.compile(r"/")
         self.root_dir = config.root
         self.book_folder = config.book_path
-        self.book_shelf = config.book_shelf
-        self._book_list_expanded = None
         self.books = None
         self.db_pointer = config.catalogue_db
         self.config = config
@@ -57,23 +60,30 @@ class Catalogue:
         :returns self._book_list_expanded: json string containing all book metadata
         """
         self.scan_folder()  # Populate file list
-        regx = re.compile(r"\.epub")
+        regx = re.compile(r"\.epub|\.mobi")
         try:
             self.books = list(filter(regx.search, filter(None, self.file_list)))
         except TypeError as e:
             print(e)
-        self._book_list_expanded = {}
+        """
-        with open(self.book_shelf, "w") as f:
         for book in self.books:
-                self._book_list_expanded[book] = self.process_book(book)
+            self._book_list_expanded[book] = self.process_by_filetype(book)
-            json.dump(self._book_list_expanded, f)
         return self._book_list_expanded
+        """
+
+    def process_by_filetype(self, book):
+        print(str(book), end='\r', flush=True)
+        if book.endswith(".epub"):
+            epub = self.process_epub(book)
+            return self.extract_metadata_epub(epub)
+        elif book.endswith(".mobi"):
+            return self.extract_metadata_mobi(book)
 
     @staticmethod
-    def process_book(book):
+    def process_epub(book):
         """Return dictionary of epub file contents"""
-        book = zipfile.ZipFile(book, "r")
         details = {}
+        book = zipfile.ZipFile(book, "r")
         with book as book_zip:
             details["files"] = []
             details["path"] = book.filename
@@ -86,7 +96,7 @@ class Catalogue:
                     details["files"].append(match.string)
         return details
 
-    def extract_metadata(self, book):
+    def extract_metadata_epub(self, book):
         """
         Return extracted metadata and cover picture
         book['path'] == Full path to ebook file
@@ -94,24 +104,123 @@ class Catalogue:
         """
         book_zip = zipfile.ZipFile(book["path"], "r")
         with book_zip as f:
-            content = self.extract_content(book_zip, book)
+            content = self.extract_content(f, book)
             soup = BeautifulSoup(content, "lxml")
             title = soup.find("dc:title")
             if title is None:
                 title = book["path"].split("/")[-1].rsplit(".", 1)[0]
             else:
                 title = title.contents[0]
+            if re.match(self.title_sanitization_regx, title):
+                if re.match(self.title_sanitization_lvl2_regx, title):
+                    title = re.split(r"-+\W", title)[1]
+                else: title = re.split(self.title_sanitization_regx, title)[2]
+
             author = soup.find("dc:creator")
             if author is not None:
                 author = author.contents[0]
             try:
-                cover = self.extract_cover_image(book_zip, book)
+                cover = self.extract_cover_image(f, book)
             except IndexError:
                 # cover = self.extract_cover_html(book_zip, book)
                 cover = DuckDuckGo().image_result(title)
-            book_details = [title, author, cover, book["path"]]
+            try:
+                description = self.stripTags(soup.find("dc:description").text)
+            except AttributeError:
+                description = None
+            try:
+                identifier = self.stripTags(soup.find("dc:identifier").text)
+            except AttributeError:
+                identifier = None
+            try:
+                publisher = self.stripTags(soup.find("dc:publisher").text)
+            except AttributeError:
+                publisher = None
+            try:
+                date = self.stripTags(soup.find("dc:date").text)
+            except AttributeError:
+                date = None
+            try:
+                rights = self.stripTags(soup.find("dc:rights").text)
+            except AttributeError:
+                rights = None
+            try:
+                tags = soup.find_all("dc:subject")
+            except AttributeError:
+                tags = None
+            ftags = None
+            if tags is not None:
+                for tag in tags:
+                    if ftags is None:
+                        ftags = tag.text
+                    else:
+                        ftags = ftags + "," + tag.text
+            book_details = [
+                title,
+                author,
+                cover,
+                book["path"],
+                description,
+                identifier,
+                publisher,
+                date,
+                rights,
+                ftags,
+            ]
         return book_details
 
+    @staticmethod
+    def stripTags(source):
+        p = re.compile(r"<.*?>")
+        return p.sub("", source)
+
+    def extract_metadata_mobi(self, book):
+        book = Mobi(book)
+        book.parse()
+        try:
+            cover_image = book.readImageRecord(0)
+        except KeyError:
+            cover_image = None
+        title = book.title().decode("utf-8")
+        author = book.author().decode("utf-8")
+        book_config = book.config
+        try:
+            description = self.stripTags(book_config['exth']['records'][103].decode("utf-8"))
+        except KeyError:
+            description = None
+        try:
+            identifier = book_config['exth']['records'][104].decode("utf-8")
+        except KeyError:
+            identifier = None
+        try:
+            publisher = book_config['exth']['records'][101].decode("utf-8")
+        except KeyError:
+            publisher = None
+        date = None
+        rights = None
+        try:
+            ftags = book_config['exth']['records'][105].decode("utf-8")
+            if ":" in ftags:
+                ftags = ftags.replace(":", ",")
+            elif ";" in ftags:
+                ftags = ftags.replace(";", ",")
+            # elif re.search(r"\s", ftags):  # Must be final assignment to avoid spliting on multiple delimeters
+            #    ftags = ftags.replace(" ", ",")
+        except KeyError:
+            ftags = None
+        return [
+            title,
+            author,
+            cover_image,
+            book.f.name,
+            description,
+            identifier,
+            publisher,
+            date,
+            rights,
+            ftags,
+        ]
+
     def extract_content(self, book_zip, book):
         """
         Opens epub as zip file filters then stores as list any files matching opf_regx
@@ -161,12 +270,13 @@ class Catalogue:
         Gets a list of new files via compare_shelf_current.
         Iterates over list and inserts new books into database.
         """
+        # TODO Refactor metadata extraction into process_book \
+        # call to more easily handle additional formats
         book_list = self.compare_shelf_current()
         db = Storage(self.config)
         for book in book_list:
-            book = self.process_book(book)
+            book = self.process_by_filetype(book)
-            extracted = self.extract_metadata(book)
+            db.insert_book(book)
-            db.insert_book(extracted)
         inserted = db.commit()
         if inserted is not True:
             print(inserted)

src/backend/lib/storage.py

@@ -1,6 +1,6 @@
 #!/usr/bin/python
 import re
-
+import datetime
 import psycopg2
 
 
@@ -53,7 +53,7 @@ class Storage:
         Insert book in database
         :returns: True if succeeds False if not
         """
-        q = "INSERT INTO books (title, author, cover, progress, file_name, pages) values (%s, %s, %s, 0, %s, 0);"
+        q = "INSERT INTO books (title, author, cover, progress, file_name, pages, description, identifier, publisher, date, rights, tags) values (%s, %s, %s, 0, %s, 0, %s, %s, %s, %s, %s, %s);"
         try:
             try:
                 cover_image = book[2].data
@@ -61,11 +61,27 @@ class Storage:
                 cover_image = book[2]
             if not book[2]:  # If cover image is missing unset entry
                 cover_image = None
-            self.cursor.execute(q, (book[0], book[1], cover_image, book[3]))
+            self.cursor.execute(
+                q,
+                (
+                    book[0],  # title
+                    book[1],  # author
+                    cover_image,
+                    book[3],  # file
+                    book[4],  # descr
+                    book[5],  # ident
+                    book[6],  # publisher
+                    datetime.datetime.now(), 
+                    book[8],  # rights
+                    book[9],  # tags
+                ),
+            )
             return True
         except Exception as e:
-            print(e)
+            if e.pgcode == '22007':  # psycopg2's error code for invalid date
-            return False
+                book[7] = psycopg2.Date(int(book[7]), 1, 1)
+                self.insert_book(book)
+            raise e
 
     def book_paths_list(self):
         """
@@ -106,8 +122,11 @@ class Storage:
             path = self.config.book_path + "/"
             _collections = []
             _pathing = book[1].split(path)[1].split("/")
+            try:
                 _pathing.pop(0)
                 _pathing.pop(-1)
+            except IndexError:
+                continue
             for _p in _pathing:
                 _s = _p.replace("'", "")
                 _x = re.sub(_title_regx, "", _s)

src/backend/pyShelf_MakeCollections.py

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 import os
 import sys
 import time

src/backend/pyShelf_ScanLibrary.py

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 import os
 import sys
 import time

src/frontend/settings.py

@@ -51,8 +51,9 @@ INSTALLED_APPS = [
     "interface",
     "interface.templatetags",
     "debug_toolbar",
+    "widget_tweaks"
 ]
-
+AUTH_USER_MODEL = "interface.User"
 MIDDLEWARE = [
     "django.middleware.security.SecurityMiddleware",
     "django.contrib.sessions.middleware.SessionMiddleware",
@@ -103,6 +104,10 @@ DATABASES = {
         "PORT": CONFIG.db_port,
     }
 }
+# Session
+# Uncomment below to enable sessions management by a memcache server
+# https://docs.djangoproject.com/en/3.0/topics/http/sessions/
+# SESSION_ENGINE = "django.contrib.sessions.backends.cached_db"
 
 # Password validation
 # https://docs.djangoproject.com/en/2.2/ref/settings/#auth-password-validators
@@ -133,6 +138,6 @@ USE_TZ = True
 
 # Static files (CSS, JavaScript, Images)
 # https://docs.djangoproject.com/en/2.2/howto/static-files/
-
+LOGIN_REDIRECT_URL = 'home'
 STATIC_URL = "/static/"
 STATIC_ROOT = os.path.join(BASE_DIR, "interface/static/")

src/frontend/urls.py

@@ -16,21 +16,56 @@ Including another URLconf
 from django.conf import settings
 from django.contrib import admin
 from django.urls import include, path, re_path
+from django.contrib.auth import views as auth_views
+from django.contrib.auth.models import User
+from django.shortcuts import HttpResponse
 from interface import views
 
 urlpatterns = [
     path("admin/", admin.site.urls),
     path("", views.index, name="index"),
+    path("home", views.home, name="home"),
+    re_path("^live", views.live, name="liverequest"),
+    path("sort/<_order>", views.index, name="index"),
+    path("flip_sort/<_order>", views.flip_sort, name="index"),
     path("download/<pk>", views.download, name="download"),
+    path("share/<pk>", views.share, name="share"),
+    path("share/<pk>", views.info, name="info"),
     path("prev_page/<bookset>", views.prev_page, name="prev_page"),
     path("next_page/<bookset>", views.next_page, name="next_page"),
-    path("search/", views.search, name="search"),
+    path("prev_page/<bookset>/<_order>", views.prev_page, name="prev_page"),
-    path("search/<query>", views.search, name="search"),
+    path("next_page/<bookset>/<_order>", views.next_page, name="next_page"),
-    path("search/<query>/<_set>", views.search, name="search"),
+    path("search/", views.index, name="search"),
+    path("search/<query>", views.index, name="search"),
+    path("search/<query>/<_set>", views.index, name="search"),
+    path("collections", views.collectionspage, name="collections"),
+    path("show_collection/<query>/<_set>", views.show_collection, name="show_collection"),
+    path("signup", views.signup, name="signup"),
+    path("login", views.userlogin, name="login"),
+    path('logout', views.userlogout, name='logout'),
+    path('favorite/<pk>', views.favorite, name='favorite'),
+    path('favorites', views.favorites, name='favorites'),
+    path('favorites/<bookset>', views.favorites, name='favorites'),
+    path('favorites/<bookset>/<query>', views.favorites, name='favorites'),
     path(
-        "show_collection/<_collection>/<_colset>",
+        'admin/password_reset/',
-        views.show_collection,
+        auth_views.PasswordResetView.as_view(),
-        name="show_collection",
+        name='admin_password_reset',
+    ),
+    path(
+        'admin/password_reset/done/',
+        auth_views.PasswordResetDoneView.as_view(),
+        name='password_reset_done',
+    ),
+    path(
+        'reset/<uidb64>/<token>/',
+        auth_views.PasswordResetConfirmView.as_view(),
+        name='password_reset_confirm',
+    ),
+    path(
+        'reset/done/',
+        auth_views.PasswordResetCompleteView.as_view(),
+        name='password_reset_complete',
     ),
 ]
 if settings.DEBUG:

src/interface/admin.py

@@ -1,6 +1,30 @@
 from django.contrib import admin
+from django.contrib.auth.admin import UserAdmin
+
+from .models import Books, Collections, Favorites, Navigation, User
+from .forms import CustomUserCreationForm, CustomUserChangeForm
+
+
+class CustomUserAdmin(UserAdmin):
+    model = User
+    add_form = CustomUserCreationForm
+    form = CustomUserChangeForm
+    list_display = ["email", "username", "facebook", "twitter", "sponsorid", "matrixid"]
+    fieldsets = UserAdmin.fieldsets + (
+        (None, {"fields": ()}),
+        ("Personal info", {"fields": ("facebook", "twitter", "matrixid")}),
+        ("Permissions", {"fields": ("sponsorid",)}),
+    )
+    # add_fieldsets is not a standard ModelAdmin attribute. UserAdmin
+    # overrides get_fieldsets to use this attribute when creating a user.
+    add_fieldsets = UserAdmin.add_fieldsets + (
+        (None, {"classes": ("wide",), "fields": ("facebook", "twitter", "sponsorid", "matrixid")},),
+    )
 
-from .models import Books
 
 # Register your models here.
 admin.site.register(Books)
+admin.site.register(Collections)
+admin.site.register(Favorites)
+admin.site.register(Navigation)
+admin.site.register(User, CustomUserAdmin)

src/interface/forms.py

@@ -0,0 +1,64 @@
+from django import forms
+from django.contrib.auth.forms import UserCreationForm, UserChangeForm, AuthenticationForm
+from .models import User
+
+
+class CustomUserCreationForm(UserCreationForm):
+    class Meta:
+        model = User
+        fields = ("username", "email", "facebook", "twitter", "sponsorid", "matrixid")
+
+
+class CustomUserChangeForm(UserChangeForm):
+    class Meta:
+        model = User
+        fields = ("username", "email", "facebook", "twitter", "sponsorid", "matrixid")
+
+
+class CustomUserLoginForm(AuthenticationForm):
+    class Meta:
+        Model = User
+        fields = ("username", "password")
+
+
+class SignUpForm(CustomUserCreationForm):
+    def __init__(self, *args, **kwargs):
+        super(CustomUserCreationForm, self).__init__(*args, **kwargs)
+        for fieldname in ['password1']:
+            self.fields[fieldname].help_text = 'At least 8 chars.'
+            self.fields[fieldname].initial = 'Password'
+        for fieldname in ['password2']:
+            self.fields[fieldname].help_text = ''
+            self.fields[fieldname].initial = 'Confirm Pass'
+
+    username = forms.CharField(
+        max_length=30,
+        required=True,
+        help_text='Required',
+        initial="",
+        label="Username",
+
+    )
+    email = forms.EmailField(
+        max_length=254,
+        help_text='Required',
+        initial="",
+        label="Email"
+    )
+    matrixid = forms.CharField(
+        max_length=30,
+        required=False,
+        help_text='Optional',
+        initial="",
+        label="Matrix Id"
+    )
+    password1 = forms.PasswordInput()
+    class Meta:
+        model = User
+        fields = ("username", "email", "matrixid")
+
+
+class UserLoginForm(CustomUserLoginForm):
+
+    class Meta:
+        model = User

src/interface/migrations/0001_initial.py

@@ -1,35 +1,111 @@
-# Generated by Django 2.2.7 on 2019-11-28 19:24
+# Generated by Django 3.0.7 on 2020-07-23 16:01
 
+from django.conf import settings
+import django.contrib.auth.models
+import django.contrib.auth.validators
 from django.db import migrations, models
+import django.db.models.deletion
+import django.utils.timezone
 
 
 class Migration(migrations.Migration):
 
     initial = True
 
-    dependencies = []
+    dependencies = [
+        ('auth', '0011_update_proxy_permissions'),
+    ]
 
     operations = [
         migrations.CreateModel(
-            name="Books",
+            name='User',
             fields=[
-                (
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                    "id",
+                ('password', models.CharField(max_length=128, verbose_name='password')),
-                    models.AutoField(
+                ('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
-                        auto_created=True,
+                ('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
-                        primary_key=True,
+                ('username', models.CharField(error_messages={'unique': 'A user with that username already exists.'}, help_text='Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.', max_length=150, unique=True, validators=[django.contrib.auth.validators.UnicodeUsernameValidator()], verbose_name='username')),
-                        serialize=False,
+                ('first_name', models.CharField(blank=True, max_length=30, verbose_name='first name')),
-                        verbose_name="ID",
+                ('last_name', models.CharField(blank=True, max_length=150, verbose_name='last name')),
-                    ),
+                ('email', models.EmailField(blank=True, max_length=254, verbose_name='email address')),
-                ),
+                ('is_staff', models.BooleanField(default=False, help_text='Designates whether the user can log into this admin site.', verbose_name='staff status')),
-                ("title", models.CharField(max_length=255)),
+                ('is_active', models.BooleanField(default=True, help_text='Designates whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
-                ("author", models.CharField(max_length=255, null=True)),
+                ('date_joined', models.DateTimeField(default=django.utils.timezone.now, verbose_name='date joined')),
-                ("categories", models.CharField(max_length=255, null=True)),
+                ('facebook', models.CharField(max_length=255, null=True)),
-                ("cover", models.BinaryField(editable=True, null=True)),
+                ('twitter', models.CharField(max_length=255, null=True)),
-                ("pages", models.IntegerField(null=True)),
+                ('ulvl', models.IntegerField(default=1)),
-                ("progress", models.IntegerField(null=True)),
+                ('sponsorid', models.IntegerField(null=True)),
-                ("file_name", models.CharField(max_length=255)),
+                ('matrixid', models.CharField(max_length=255, null=True)),
+                ('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.Group', verbose_name='groups')),
+                ('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.Permission', verbose_name='user permissions')),
             ],
-            options={"db_table": "books",},
+            options={
+                'verbose_name': 'user',
+                'verbose_name_plural': 'users',
+                'abstract': False,
+            },
+            managers=[
+                ('objects', django.contrib.auth.models.UserManager()),
+            ],
+        ),
+        migrations.CreateModel(
+            name='Books',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('title', models.TextField()),
+                ('author', models.CharField(max_length=255, null=True)),
+                ('categories', models.TextField(null=True)),
+                ('cover', models.BinaryField(editable=True, null=True)),
+                ('pages', models.IntegerField(null=True)),
+                ('progress', models.IntegerField(null=True)),
+                ('file_name', models.TextField()),
+                ('description', models.TextField(null=True)),
+                ('identifier', models.CharField(max_length=255, null=True)),
+                ('publisher', models.TextField(null=True)),
+                ('date', models.DateField(null=True)),
+                ('rights', models.CharField(max_length=255, null=True)),
+                ('tags', models.TextField(null=True)),
+            ],
+            options={
+                'db_table': 'books',
+            },
+        ),
+        migrations.CreateModel(
+            name='Navigation',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('title', models.CharField(max_length=255)),
+                ('link', models.CharField(max_length=255, null=True)),
+                ('category', models.CharField(max_length=255, null=True)),
+                ('parent_id', models.IntegerField(null=True)),
+                ('alt', models.CharField(max_length=255, null=True)),
+                ('type', models.IntegerField(null=True)),
+                ('socket', models.CharField(max_length=255)),
+            ],
+            options={
+                'db_table': 'navigation',
+            },
+        ),
+        migrations.CreateModel(
+            name='Favorites',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('book', models.ForeignKey(default=None, on_delete=django.db.models.deletion.CASCADE, to='interface.Books')),
+                ('user', models.ForeignKey(default=None, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'db_table': 'favorites',
+            },
+        ),
+        migrations.CreateModel(
+            name='Collections',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('collection', models.CharField(max_length=255)),
+                ('book_id', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='interface.Books')),
+            ],
+            options={
+                'db_table': 'collections',
+            },
         ),
     ]

src/interface/migrations/0002_auto_20200101_0445.py

@@ -1,14 +0,0 @@
-# Generated by Django 2.2.7 on 2020-01-01 04:45
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ("interface", "0001_initial"),
-    ]
-
-    operations = [
-        migrations.AlterModelOptions(name="books", options={"managed": False},),
-    ]

src/interface/migrations/0003_auto_20200101_0447.py

@@ -1,14 +0,0 @@
-# Generated by Django 2.2.7 on 2020-01-01 04:47
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ("interface", "0002_auto_20200101_0445"),
-    ]
-
-    operations = [
-        migrations.AlterModelOptions(name="books", options={},),
-    ]

src/interface/migrations/0004_collections.py

@@ -1,37 +0,0 @@
-# Generated by Django 3.0.2 on 2020-02-04 20:22
-
-import django.db.models.deletion
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ("interface", "0003_auto_20200101_0447"),
-    ]
-
-    operations = [
-        migrations.CreateModel(
-            name="Collections",
-            fields=[
-                (
-                    "id",
-                    models.AutoField(
-                        auto_created=True,
-                        primary_key=True,
-                        serialize=False,
-                        verbose_name="ID",
-                    ),
-                ),
-                ("collection", models.CharField(max_length=255)),
-                (
-                    "book_id",
-                    models.ForeignKey(
-                        on_delete=django.db.models.deletion.PROTECT,
-                        to="interface.Books",
-                    ),
-                ),
-            ],
-            options={"db_table": "collections",},
-        ),
-    ]

src/interface/models.py

@@ -1,5 +1,8 @@
 from django.contrib.postgres.search import SearchVector
 from django.db import models
+from django.conf import settings
+from django.contrib.auth.models import AbstractUser, User
+from django.contrib.auth import get_user_model
 
 # Create your models here.
 
@@ -22,30 +25,43 @@ class Books(models.Model):
     def __str__(self):
         return self.title
 
-    title = models.CharField(max_length=255)
+    title = models.TextField(max_length=None)
     author = models.CharField(max_length=255, null=True)
-    categories = models.CharField(max_length=255, null=True)
+    categories = models.TextField(max_length=None, null=True)
     cover = models.BinaryField(null=True, editable=True)
     pages = models.IntegerField(null=True)
     progress = models.IntegerField(null=True)
-    file_name = models.CharField(max_length=255, null=False)
+    file_name = models.TextField(max_length=None, null=False)
+    date = models.DateTimeField(auto_now_add=True)
+    description = models.TextField(null=True)
+    identifier = models.CharField(max_length=255, null=True)
+    publisher = models.TextField(max_length=None, null=True)
+    date = models.DateField(null=True)
+    rights = models.CharField(max_length=255, null=True)
+    tags = models.TextField(max_length=None, null=True)
 
     def generic_search(self, query):
         try:
             results = Books.objects.annotate(
-                search=SearchVector("title", "file_name", "author"),
+                search=SearchVector("title", "file_name", "author", "tags"),
             ).filter(search=query)
         except Exception as e:
             raise
         return results
 
+    def search_by_collection(self, query):
+        try:
+            return Books.objects.filter(categories=query)
+        except Exception as e:
+            raise
+
 
 class Collections(models.Model):
     class Meta:
         db_table = "collections"
 
     def __str__(self):
-        return self.collection
+        return self.collection.__str__()
 
     collection = models.CharField(max_length=255)
     book_id = models.ForeignKey(Books, on_delete=models.PROTECT)
@@ -55,10 +71,109 @@ class Collections(models.Model):
         return reverse("model-detail-view", args=[str(self.id)])
 
     def generic_search(self, query):
+        books =[]
         try:
-            results = Books.objects.annotate(search=SearchVector("collection"),).filter(
+            #results = Collections.objects.all().filter(
+            #    collection=query
+            #)
+            results = Collections.objects.prefetch_related('book_id').annotate(search=SearchVector("collection"), ).filter(
                 search=query
             )
         except Exception as e:
             raise
+        _results = results.values('book_id')
+        for r in results:
+            books.append(
+                {
+                    "pk": r.book_id.id,
+                    "title": r.book_id.title,
+                    "author": r.book_id.author,
+                    "categories": r.book_id.categories,
+                    "cover": r.book_id.cover,
+                    "pages": r.book_id.pages,
+                    "progress": r.book_id.progress,
+                    "file_name": r.book_id.file_name,
+                    "date": r.book_id.date,
+                    "description": r.book_id.description,
+                    "identifier": r.book_id.identifier,
+                    "publisher": r.book_id.publisher,
+                    "date": r.book_id.date,
+                    "rights": r.book_id.rights,
+                    "tags": r.book_id.tags
+                }
+            )
+        return books
+
+
+class Navigation(models.Model):
+    """
+    pyShelfs Navigation Database class
+    :param title: Link Text
+    :param link: Link link :)
+    :param category: Where in the nav tree do I belong
+    :param parent_id: This link is a sub link of link with id of me
+    :param alt: Alternate text of link
+    :param type: Web link, or Socket link which will be expected to act on \
+    the link, and the action defined in socket
+    :param socket: if a Socket link define socket here
+    """
+
+    class Meta:
+        db_table = "navigation"
+
+    def __str__(self):
+        return self.title
+
+    title = models.CharField(max_length=255)
+    link = models.CharField(max_length=255, null=True)
+    category = models.CharField(max_length=255, null=True)
+    parent_id = models.IntegerField(null=True, editable=True)
+    alt = models.CharField(max_length=255, null=True)
+    type = models.IntegerField(null=True)
+    socket = models.CharField(max_length=255, null=False)
+
+    def generic_search(self, query):
+        try:
+            results = Navigation.objects.annotate(
+                search=SearchVector("title", "parent_id", "category"),
+            ).filter(search=query)
+        except Exception as e:
+            raise
         return results
+
+
+class User(AbstractUser):
+    facebook = models.CharField(max_length=255, null=True)
+    twitter = models.CharField(max_length=255, null=True)
+    ulvl = models.IntegerField(default=1)
+    sponsorid = models.IntegerField(null=True)
+    matrixid = models.CharField(max_length=255, null=True)
+
+    def __str__(self):
+        return self.username
+
+
+class Favorites(models.Model):
+    """
+     Favorites Database class
+    :param book: book foreign key
+    :param user: user foreign key
+    """
+
+    class Meta:
+        db_table = "favorites"
+
+    def __str__(self):
+        return str(self.book)
+ 
+    book = models.ForeignKey(Books, on_delete=models.CASCADE, default=None)
+    user = models.ForeignKey(User, on_delete=models.CASCADE, default=None)
+
+    def generic_search(self, query):
+        try:
+            results = Favorites.objects.annotate(search=SearchVector("user"),).filter(search=query)
+        except Exception as e:
+            raise
+        return results
+
+