Python

importerror: attempted relative import with no known parent package

I want to import a function from another file in the same directory.

Sometimes it works for me  from .mymodule import myfunction but sometimes I get a:

SystemError: Parent module '' not loaded, cannot perform relative import

Sometimes it works with from mymodule import myfunction, but sometimes I also get a:

SystemError: Parent module '' not loaded, cannot perform relative import

I don’t understand the logic here and I couldn’t find any explanation. This looks completely random.

Could someone explain to me what’s the logic behind all this?

 

Answer:

 

unfortunately, this module needs to be inside the package, and it also needs to be runnable as a script, sometimes. Any idea how I could achieve that?

It’s quite common to have a layout like this…

main.py
mypackage/
    __init__.py
    mymodule.py
    myothermodule.py

…with a mymodule.py like this…

#!/usr/bin/env python3

# Exported function
def as_int(a):
    return int(a)

# Test function for module  
def _test():
    assert as_int('1') == 1

if __name__ == '__main__':
    _test()

…a myothermodule.py like this…

#!/usr/bin/env python3

from .mymodule import as_int

# Exported function
def add(a, b):
    return as_int(a) + as_int(b)

# Test function for module  
def _test():
    assert add('1', '1') == 2

if __name__ == '__main__':
    _test()

…and a main.py like this…

#!/usr/bin/env python3

from mypackage.myothermodule import add

def main():
    print(add('1', '1'))

if __name__ == '__main__':
    main()

…which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import…

from .mymodule import as_int

The way you’re supposed to run it is…

python3 -m mypackage.myothermodule

…but it’s somewhat verbose and doesn’t mix well with a shebang line like #!/usr/bin/env python3.

The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use…

from mymodule import as_int

…although, if it’s not unique, or your package structure is more complex, you’ll need to include the directory containing your package directory in PYTHONPATH, and do it like this…

from mypackage.mymodule import as_int

…or if you want it to work “out of the box”, you can frob the PYTHONPATH in code first with this…

import sys
import os

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))

from mypackage.mymodule import as_int

It’s kind of a pain, but there’s a clue as to why in an email written by a certain Guido van Rossum…

I’m -1 on this and on any other proposed twiddlings of the __main__ machinery. The only use case seems to be running scripts that happen to be living inside a module’s directory, which I’ve always seen as an antipattern. To make me change my mind you’d have to convince me that it isn’t.

Whether running scripts inside a package is an antipattern or not is subjective, but personally, I find it really useful in a package I have that contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.

Python Math – TypeError: ‘NoneType’ object is not subscriptable

I’m making a small program for math (no particular reason, just kind of wanted to) and I ran into the error “TypeError: ‘NoneType’ object is not subscriptable.

I have never before seen this error, so I have no idea what it means.

import math

print("The format you should consider:")
print str("value 1a")+str(" + ")+str("value 2")+str(" = ")+str("value 3a ")+str("value 4")+str("\n")

print("Do not include the letters in the input, it automatically adds them")

v1 = input("Value 1: ")
v2 = input("Value 2: ")
v3 = input("Value 3: ")
v4 = input("Value 4: ")

lista = [v1, v3]
lista = list.sort(lista)

a = lista[1] - lista[0]

list = [v2, v4]
list = list.sort(list)

b = list[1] = list[0]

print str(a)+str("a")+str(" = ")+str(b)

The error:

Traceback (most recent call last):
  File "C:/Users/Nathan/Documents/Python/New thing", line 16, in <module>
    a = lista[1] - lista[0]
TypeError: 'NoneType' object is not subscriptable,

valueerror: too many values to unpack (expected 2)


Answer:

lista = list.sort(lista)

This should be

lista.sort()

The .sort() method is in-place, and returns None. If you want something not in-place, which returns a value, you could use

sorted_list = sorted(lista)

Aside #1: please don’t call your lists list. That clobbers the builtin list type.

Aside #2: I’m not sure what this line is meant to do:

print str("value 1a")+str(" + ")+str("value 2")+str(" = ")+str("value 3a ")+str("value 4")+str("\n")

is it simply

print "value 1a + value 2 = value 3a value 4"

? In other words, I don’t know why you’re calling str on things which are already str.

Aside #3: sometimes you use print("something") (Python 3 syntax) and sometimes you use print "something" (Python 2). The latter would give you a SyntaxError in py3, so you must be running 2.*, in which case you probably don’t want to get in the habit or you’ll wind up printing tuples, with extra parentheses. I admit that it’ll work well enough here, because if there’s only one element in the parentheses it’s not interpreted as a tuple, but it looks strange to the pythonic eye..

Errors such as: ‘Solving environment: failed with initial frozen solve. Retrying with flexible solve’ & ‘unittest’ tab

I am working with spyder – python. I want to test my codes. I have followed the pip install spyder-unittest and pip install pytest. I have restarted the kernel and restarted my MAC as well. Yet, the Unit Testing tab does not appear. Even when I drop down Run cannot find the Run Unit test. Does someone know how to do this?

So, I solved the issue by running the command:

conda config --set channel_priority false.

And then proceeded with the unittest download with the command run:

conda install -c spyder-ide spyder-unittest.

The first command run conda config --set channel_priority false may solve other issues such as:

Solving environment: failed with initial frozen solve. Retrying with flexible solve

ImportError: cannot import name ‘_unicodefun’ from ‘click’

if running our lint checks with the python black pkg. an error comes up

ImportError: cannot import name '_unicodefun' from 'click' (/Users/robot/.cache/pre-commit/repo3u71ccm2/py_env-python3.9/lib/python3.9/site-packages/click/__init__.py)

related issues:

https://github.com/psf/black/issues/2976
https://github.com/dask/distributed/issues/6013

 

Answer 1

This has been fixed by Black 22.3.0. Versions before that won’t work with click 8.1.0.

https://github.com/psf/black/issues/2964

E.g.: black.yml

 

          python-version: 3.8
      - name: install black
        run: |
-          pip install black==20.8b1
+          pip install black==22.3.0
      - name: run black
        run: |
          black . --check --line-length 100

https://github.com/Clinical-Genomics/cgbeacon2/pull/221/files

As a workaround pin click to last version via pip install --upgrade click==8.0.2

Running a fine-tune model for my CNN : Value Error

So I am trying to use a pre-trained model on my data set to then compare it to my own cnn model. However, I see an error as soon as I try to do model. fit so much that ((None, 4, 4, 1) vs (None,)). Where is this error coming from? Am I supposed to edit the pre-tune cnn.

The model that I am using is ResNET50 with no modification except the input layer changed to 128 and there are 2 outputs.

Any help is welcome,

CODE:
history = modelB.fit_generator(train_data,
validation_data = test_data,
epochs=5,
steps_per_epoch = 1714,)

ERROR

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
2 validation_data = test_data,
3 epochs=5,
----> 4 steps_per_epoch = 1714,)

2 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
1145 except Exception as e: # pylint:disable=broad-except
1146 if hasattr(e, "ag_error_metadata"):
-> 1147 raise e.ag_error_metadata.to_exception(e)
1148 else:
1149 raise

ValueError: in user code:

File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step **
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 919, in compute_loss
y, y_pred, sample_weight, regularization_losses=self.losses)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 201, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 141, in __call__
losses = call_fn(y_true, y_pred)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 245, in call **
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 1932, in binary_crossentropy
backend.binary_crossentropy(y_true, y_pred, from_logits=from_logits),
File "/usr/local/lib/python3.7/dist-packages/keras/backend.py", line 5247, in binary_crossentropy
return tf.nn.sigmoid_cross_entropy_with_logits(labels=target, logits=output)

ValueError: `logits` and `labels` must have the same shape, received ((None, 4, 4, 1) vs (None,)).

Answer 1:

The Issue is with the loss function used when you compile the model.

Replace the compile with below code:

model.compile(optimizer='adam',loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])

Answer 2:

Use tf.keras.utils.plot_model to print out a graphic representation of the model. you have a mismatch between the number of input and output nodes.

ImportError: cannot import name ‘safe_str_cmp’ from ‘werkzeug.security’

The problem

any ideas why this error?

my project was working fine, i copied it to an external drive and onto my laptop to work on the road, it worked fine. i copied back to my desktop and had a load of issues with invalid interpreters etc, so i made a new project and copied just the scripts in, made a new requirements.txt and installed all the packages, but when i run i get this error

Traceback (most recent call last):
File "E:\Dev\spot_new\flask_blog\run.py", line 1, in
from flaskblog import app
File "E:\Dev\spot_new\flask_blog\flaskblog\__init__.py", line 3, in
from flask_bcrypt import Bcrypt
File "E:\Dev\spot_new\venv\lib\site-packages\flask_bcrypt.py", line 21, in
from werkzeug.security import safe_str_cmp
ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security' (E:\Dev\spot_new\venv\lib\site-packages\werkzeug\security.py)

i’ve tried uninstalling python, anaconda, pycharm, deleting every reg key and environment variable i can find that looks pythonic, reinstalling all from scratch but still no dice.

any input greatly appreciated!

thanks

Answer 1:

Werkzeug released v2.1.0 today, removing werkzeug.security.safe_str_cmp.

You can probably resolve this issue by pinning Werkzeug~=2.0.0 in your requirements.txt file (or similar).

Share

ImportError: cannot import name ‘Markup’ from ‘jinja2’

When I recently deployed my project that includes Flask==1.0.2 and Jinja2>=2.10.1, I got the following error. It was running fine when I deployed it the previous day. I tried updating Jinja2 but that didn’t fix the issue.

 

requirements.txt:

attrs>=19.1.0
boto3==1.17.23
click==7.0
itsdangerous==2.0.1
flasgger==0.9.5
Flask==1.0.2
Flask-RESTful>=0.3.6
flask-cors==3.0.8
gunicorn==20.1.0
Jinja2>=2.10.1
jsonschema>=3.0.1,<4.0
marshmallow>=3.0,<=3.6
marshmallow3-annotations>=1.0.0
pytz==2021.1
requests>=2.25.0
requests-aws4auth==1.1.0
statsd==3.2.1
typing==3.6.4
werkzeug>=2.0.0
wheel==0.36.2
itsdangerous==2.0.1 Answer 1:

Version 3.0.1

Regarding the documentation

Fixed calling deprecated jinja2.Markup without an argument. Use markupsafe.Markup instead. #1438

So to import Markup use the following code :

>>> from jinja2.utils import markupsafe 
>>> markupsafe.Markup()
Markup('')

Answer 2:

As the import error comes from flask File "/usr/local/lib/python3.7/site-packages/flask/__init__.py

I tried to create a new Flask application using Flask==1.0.2 and found that the error comes from this version of Flask when it is used with Jinja2>=2.10.1.

But when you remove Flask==1.0.2 and install Flask==2.0.3, everything works fine.

pip uninstall  Flask Jinja2
pip install Flask Jinja2

Dependencies

pip freeze
click==8.0.4
Flask==2.0.3
itsdangerous==2.1.2
Jinja2==3.1.1
MarkupSafe==2.1.1
Werkzeug==2.0.3

What does yield do in python

Shortcut to understanding yield

When you see a function with yield statements, apply this easy trick to understand what will happen:

  1. Insert a line result = [] at the start of the function.
  2. Replace each yield expr with result.append(expr).
  3. Insert a line return result at the bottom of the function.
  4. Yay – no more yield statements! Read and figure out code.
  5. Compare function to the original definition.

This trick may give you an idea of the logic behind the function, but what actually happens with yield is significantly different than what happens in the list-based approach. In many cases, the yield approach will be a lot more memory efficient and faster too. In other cases, this trick will get you stuck in an infinite loop, even though the original function works just fine. Read on to learn more…

Don’t confuse your Iterables, Iterators, and Generators

First, the iterator protocol – when you write


for x in mylist:
...loop body...

Python performs the following two steps:

 

  1. Gets an iterator for mylist:
  2. Call iter(mylist) -> this returns an object with a next() method (or __next__() in Python 3).
  3. [This is the step most people forget to tell you about]

 

  1. Uses the iterator to loop over items:
  2. Keep calling the next() method on the iterator returned from step 1. The return value from next() is assigned to x and the loop body is executed. If an exception StopIteration is raised from within next(), it means there are no more values in the iterator and the loop is exited.

The truth is Python performs the above two steps anytime it wants to loop over the contents of an object – so it could be a for loop, but it could also be code like otherlist.extend(mylist) (where otherlist is a Python list).

Here mylist is iterable because it implements the iterator protocol. In a user-defined class, you can implement the __iter__() method to make instances of your class iterable. This method should return an iterator. An iterator is an object with a next() method. It is possible to implement both __iter__() and next() on the same class, and have __iter__() return self. This will work for simple cases, but not when you want two iterators looping over the same object at the same time.

So that’s the iterator protocol, many objects implement this protocol:

  1. Built-in lists, dictionaries, tuples, sets, files.
  2. User-defined classes that implement __iter__().
  3. Generators.

Note that a for loop doesn’t know what kind of object it’s dealing with – it just follows the iterator protocol, and is happy to get item after item as it calls next(). Built-in lists return their items one by one, dictionaries return the keys one by one, files return the lines one by one, etc. And generators return… well that’s where yield comes in:


def f123():
yield 1
yield 2
yield 3

for item in f123():
print item

Instead of yield statements, if you had three return statements in f123() only the first would get executed, and the function would exit. But f123() is no ordinary function. When f123() is called, it does not return any of the values in the yield statements! It returns a generator object. Also, the function does not really exit – it goes into a suspended state. When the for loop tries to loop over the generator object, the function resumes from its suspended state at the very next line after the yield it previously returned from, executes the next line of code, in this case, a yield statement, and returns that as the next item. This happens until the function exits, at which point the generator raises StopIteration and the loop exits.

So the generator object is sort of like an adapter – at one end it exhibits the iterator protocol, by exposing __iter__() and next() methods to keep the for loop happy. At the other end, however, it runs the function just enough to get the next value out of it, and puts it back in suspended mode.

Why Use Generators?

Usually, you can write code that doesn’t use generators but implements the same logic. One option is to use the temporary list ‘trick’ I mentioned before. That will not work in all cases, for e.g. if you have infinite loops, or it may make inefficient use of memory when you have a really long list. The other approach is to implement a new iterable class SomethingIter that keeps the state in instance members and performs the next logical step in its next() (or __next__() in Python 3) method. Depending on the logic, the code inside the next() method may end up looking very complex and be prone to bugs. Here generators provide a clean and easy solution.

How to use requirements.txt to install all dependencies in a python project

Solution 1

If you are using Linux OS:

  1. Remove matplotlib==1.3.1 from requirements.txt
  2. Try to install with sudo apt-get install python-matplotlib
  3. Run pip install -r requirements.txt (Python 2), or pip3 install -r requirements.txt (Python 3)
  4. pip freeze > requirements.txt

If you are using Windows OS:

  1. python -m pip install -U pip setuptools
  2. python -m pip install matplotlib

 

Solution 2

 

pip install -r requirements.txt for python 2.x

pip3 install -r requirements.txt for python 3.x (in case multiple versions are installed)

modulenotfounderror: no module named ‘tensorflow’

modulenotfounderror: no module named ‘tensorflow’ anaconda

The reason Python 3.5 environment is unable to import Tensorflow is that Anaconda does not store the tensorflow package in the same environment.

One solution is to create a new separate environment in Anaconda dedicated to TensorFlow with its own Spyder


conda create -n newenvt anaconda python=3.5
activate newenvt

and then install tensorflow into newenvt